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PREFACE 


This  report  documents  a  Federal  Aviation  Administration  operational 
evaluation  of  the  initial  group  of  four  air  traffic  control 
services  which  have  been  designed  for  implementation  on  the  Mode 
S  Data  Link  system.  The  report  is  organized  in  two  volumes. 

Volume  I  contains  the  main  body  of  the  report.  It  includes  a 
detailed  description  of  the  objectives  of  the  evaluation  and  of  the 
technical  approach  and  test  methods  that  were  used.  In  addition, 
the  primary  results  of  the  controller  and  aircrew  portions  of  the 
study,  conclusions,  and  recommendations  are  presented. 

Volume  II  contains  a  set  of  appendixes  to  the  report.  These 
appendixes  are  referenced  in  Volume  I  and  include  technical 
documentation  of  the  test  hardware  and  of  controller  and  pilot  Data 
Link  procedures.  The  appendixes  also  present  detailed  analyses  of 
the  data  which  were  collected  in  the  study.  These  appendixes  were 
the  basis  for  the  primary  results  sections  presented  in  Volume  I. 
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EXECUTIVE  SUMMARY 


INTRODUCTION. 

The  Federal  Aviation  Administration  (FAA)  is  pursuing  an  initiative 
to  develop  and  implement  a  Data  Link  system  intended  to  enhance 
communications  between  ground-based  air  traffic  control  (ATC)  and 
airborne  systems.  By  providing  digital  information  transfer  with 
the  ability  to  discretely  address  individual  aircraft,  Data  Link 
is  expected  to  relieve  frequency  congestion  on  existing  voice  radio 
channels  while  increasing  the  overall  safety  and  productivity  of 
the  ATC  system. 

In  order  to  insure  that  the  introduction  of  Data  Link  will  have  an 
optimal  positive  impact  on  ATC,  the  FAA  is  conducting  a  program  of 
research  to  guide  system  design  efforts,  evaluate  the  benefits  of 
Data  Link  to  ATC,  and  assess  its  effects  on  the  air  traffic 
controllers  and  aircrews  who  will  use  the  system.  Preliminary 
design  studies  completed  during  the  past  year  (Talotta,  et  al., 
1988,  1989)  defined  the  detailed  configurations  for  the  initial 
group  of  ATC  services  and  functions  scheduled  for  implementation 
on  Data  Link.  This  report  presents  the  results  of  the  first 
operational  evaluation  of  these  initial  capabilities  in  a  full 
scale,  ATC  simulation  manned  by  en  route  air  traffic  controllers. 
In  addition,  this  report  presents  the  findings  of  a  concurrent 
study  in  which  aircrews  flying  simulated  aircraft  in  the  ATC 
scenarios  evaluated  the  initial  services  and  a  preliminary  flight 
deck  display/control  device  for  Data  Link  communications. 

OBJECTIVES. 

The  general  purpose  of  the  air  traffic  controller  substudy  was  to 
assess  the  impact  of  the  initial  package  of  Data  Link  ATC  services 
and  functions  in  the  context  of  a  high  fidelity  simulation  of  en 
route  ATC  activity.  The  specific  objectives  were  to  measure  the 
beneficial  effect  of  Data  Link  on  voice  radio  frequency  congestion 
and  to  determine  the  impact  on  ATC  performance  and  effectiveness, 
controller  workload,  and  controller  acceptance.  The  aircrew 
substudy  was  designed  to  evaluate  the  overall  effectiveness  of  Data 
Link  communications  in  aircraft  operations  both  for  the  ATC 
services  uplinked  by  the  subjects  in  the  controller  substudy,  and 
fcr  a  set  of  Data  Link  weather  services  available  by  direct  aircrew 
interaction  with  simulated  data  bases. 

DATA  LINK  SERVICES. 

Two  ATC  services  and  two  general  Data  Link  functions  were  tested 
in  the  controller  substudy.  The  ATC  services  were  altitude 
assignment  and  transfer  of  communication.  Data  Link  procedures  for 
both  of  these  services  were  designed  to  closely  parallel  current 
en  route  controller  procedures  with  the  addition  of  a  minimal 
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number  of  keyboard  actions.  The  status  of  Data  Link  transactions 
with  aircraft  were  displayed  in  a  Plan  View  Display  (PVD)  list  and 
in  the  Full  Data  Block  (FDB) .  Aircraft  equipage  and  uplink 
eligibility  were  indicated  by  graphic  characters  in  the  first  field 
of  the  FDB. 

The  Data  Link  ATC  functions  were  menu  text  and  free  text.  Menu 
text  permits  standard  or  frequently  required  ATC  messages  to  be 
stored  in  a  menu  displayed  on  the  PVD  and  selected  for  uplink  as 
required.  For  the  present  study,  menu  messages  were  restricted  to 
interim  altitude  assignments,  sometimes  combined  with  fix  crossing 
restrictions.  Free  text  provides  controllers  with  the  ability  to 
engage  in  unconstrained  ground-air  communications  using  typed 
keyboard  entries.  This  function  is  envisioned  primarily  as  a  back¬ 
up  communications  capability. 

In  addition  to  the  primary  ATC  services  delivered  by  the 
controllers,  the  aircrew  substudy  also  examined  a  group  of  weather 
services  which  the  pilots  could  request  by  direct  downlink  to  a 
data  base.  These  services  were  terminal  forecast,  winds/ 
temperatures  aloft,  surface  observation,  and  Pilot  Reports 
(PIREPS) .  The  aircrew  received  and  responded  to  Data  Link  messages 
on  a  touch  sensitive  display. 

APPROACH. 

The  evaluation  was  conducted  in  the  FAA  Technical  Center's  Data 
Link  Test  Bed.  Two  30-minute  test  scenarios  presented  controllers 
with  realistic  en  route  ATC  problems  involving  four  sectors  of 
adjacent  airspace.  A  majority  of  the  aircraft  in  the  scenarios 
were  simulated  radar  targets  controlled  by  simulation  operators. 
However,  two  of  the  targets  in  each  test  run  were  driven  by  high 
fidelity  simulators  representing  a  large  transport  aircraft  and  a 
small  General  Aviation  (GA)  aircraft.  The  airliner  was  manned  by 
professional  aircrews,  while  the  small  aircraft  was  flown  by  GA 
pilots . 

The  eight  en  route  controllers  participated  in  teams  of  four  in  14 
test  runs  in  which  Data  Link  equipage  levels  of  the  air  traffic 
ranged  from  0  (voice  radio  only)  to  20  and  70  percent.  In  one  of 
the  runs,  average  uplink  and  dov'nlink  times  were  extended  to  assess 
the  impact  of  "worst  case"  Data  Link  communication  transmission 
delays.  Each  of  the  four  participating  aircrews  and  the  four  GA 
pilots  participated  in  two  Data  Link  runs  and  one  baseline,  voice- 
only  run. 

Controller  measures  included  records  of  voice  and  Data  Link  usage, 
ratings  of  controller  workload  and  service  design  acceptability, 
estimates  of  operational  effectiveness,  a  wrap-up  questionnaire, 
and  extensive  debriefing  to  elicit  required  Data  Link  design 
modifications.  Aircrew  measures  evaluated  perceptions  of  the 
effectiveness  of  the  services  and  the  usability  of  the  preliminary 
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flight  deck  Data  Link  Input/Output  (I/O)  device.  Times  required 
for  aircrew  to  detect  and  respond  to  Data  Link  and  voice  messages 
also  were  recorded. 

PRIMARY  RESULTS . 

The  controller  substudy  clearly  demonstrated  the  benefit  that  the 
initial  group  of  Data  Link  services  can  be  expected  to  have  on 
relieving  frequency  congestion  of  voice  radio  channels.  The 
availablity  of  these  functions  reduced  the  number  of  controller 
voice  transmissions  by  up  to  41  percent,  and  total  controller 
occupation  time  of  the  radio  frequencies  by  up  to  ,5  percent  (see 
figure  ES-1) .  Furthermore,  as  more  Data  Link  equipped  aircraft  were 
introduced,  there  was  an  apparent  increase  in  the  efficiency  of  the 
communication  process  as  requirements  for  repetition  of  voice 
messages  and  clarification  of  clearances  were  reduced. 

The  positive  impact  of  Data  Link  was  not  achieved  at  any  observed 
loss  in  ATC  performance  or  controller  capability.  No  critical  ATC 
incidents  were  noted  during  the  tests,  and  controller  workload 
estimates  were  statistically  identical  in  the  voice  and  Data  Link 
conditions  (figure  ES-1).  The  results  also  showed  that  Data  Link 
transmission  delays  failed  to  have  a  negative  effect  on  controller 
activities.  All  of  the  specific  designs  for  the  Data  Link  services 
were  rated  as  operationally  effective  and  acceptable  to 
controllers . 

The  results  of  the  aircrew  substudy  indicated  that  initial  pilot 
perceptions  of  Data  Link  were  positive  and  that  its  implementation 
would  enhance  the  quality  of  air-ground  communications.  Additional 
findings  indicated  that  the  average  total  times  between  receipt  of 
a  message  in  the  aircraft  and  the  crew  response  to  ATC  ranged  from 
9.8  seconds  in  the  voice  conditions  to  10.9  seconds  in  the  Data 
Link  conditions. 

The  workload  associated  with  Data  Link  operations  was  considered 
acceptable  to  pilots.  Furthermore,  while  the  aircrew  subjects 
indicated  further  development  of  display  formats  and  crew 
coordination  procedures  will  be  required,  they  felt  that  persistent 
Data  Link  displays  will  reduce  demands  on  pilot  memory  and  that 
Data  Link  will  be  capable  of  replacing  most  functions  currently 
served  by  voice  radio  in  en  route  environments. 

RECOMMENDATIONS. 


Based  on  the  results  outlined  above,  it  is  recommended  that  the 
initial  group  of  Data  Link  ATC  services  be  incorporated  as 
modifications  to  current  er  route  ATC  software  and  hardware,  and 
that  they  be  subjected  to  operational  test  and  evaluation  (OT&E) . 
Minor  outstanding  design  issues  and  potential  enhancements  revealed 
by  the  present  study  should  be  examined  by  the  Air  Traffic  Data 
Link  Validation  Team  (ATDLVT)  and  resolved.  It  is  further 
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recommended  that,  because  of  the  success  of  the  operational 
evaluation  and  the  positive  impressions  expressed  by  participating 
controllers  and  pilots,  research  and  development  work  be  expedited 
to  extend  Data  Link  applications  to  the  full  range  of  en  route  and 
terminal  AT C  services. 

Because  the  development  of  refined  flight  deck  interfaces  has 
lagged  behind  corresponding  ground  controller  work,  it  is 
recommended  that  additional  studies  be  conducted  to  facilitate  the 
integration  of  Data  Link  with  the  aircrew  task  environment. 

Finally,  in  order  to  identify  mutually  acceptable  rules  and 
procedures  which  will  govern  pilot  and  controller  communications 
using  Data  Link,  it  is  recommended  that  a  mechanism  be  created  to 
promote  relevant  interactions  between  the  ATDLVT  and  a 
representative  pilot  group. 


DATA  LINK  AND  VOICE  ME88AQE8 
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1. 


INTRODUCTION. 


1.1  PURPOSE. 


This  document  presents  the  results  of  a  Federal  Aviation 
Administration  (FAA)  operational  evaluation  study  of  the  initial 
package  of  en  route  air  traffic  control  (ATC)  services  designed  for 
implementation  in  the  Mode  S  Data  Link  system.  The  study  employed 
high  fidelity,  integrated  simulations  of  the  ATC  and  flight  deck 
environments  involving  both  en  route  air  traffic  controllers  and 
pilots.  Test  trials  were  conducted  to  compare  Data  Link 
communications  to  current  voice  radio  procedures  using  measures  of 
system  effectiveness  and  productivity  as  well  as  controller  and 
pilot  performance,  workload,  and  acceptance. 

1.2  BACKGROUND. 

1.2.1  Data  Link  Requirements. 

The  extraordinary  increase  in  commercial  and  general  air  traffic 
in  recent  years  has  begun  to  tax  the  practical  limits  of  the  ATC 
system  in  the  United  States.  Because  this  growth  is  expected  to 
continue  into  the  next  century,  the  FAA  has  devised  the  National 
Airspace  System  (NAS)  Plan  to  update  and  enhance  ATC  technology. 
Implementation  of  the  NAS  Plan  commenced  in  1982  with  the  primary 
goals  of  improving  ATC  system  safety,  increasing  its  efficiency, 
improving  the  ability  of  air  traffic  controllers  to  handle 
increasing  levels  of  traffic,  and  reducing  operational  errors. 

One  of  the  major  factors  which  constrains  the  productivity  of  the 
current  ATC  system  is  the  communication  channel  between  the  air 
traffic  controller  and  the  aircraft  pilot.  Because  this  voice 
radio  link  operates  i  a  broadcast  mode  between  a  single  controller 
and  all  aircraft  contained  within  the  airspace  under  his  control, 
frequency  congestion  rapidly  consumes  available  communications  time 
and  reduces  system  efficiency  when  the  volume  of  air  traffic 
increases.  In  addition,  the  resulting  interference  from 
simultaneous  attempts  by  multiple  pilots  to  contact  a  controller, 
and  from  other  noise  factors  common  to  very  high  frequency  ( VHF) 
voice  communications,  increase  the  possibility  of  interpretation 
errors  in  crowded  ATC  environments. 

A  key  component  of  the  NAS  Plan  that  was  developed  to  address  the 
problems  of  frequency  congestion  and  communications  errors  is  the 
ground-air  Data  Link.  In  general,  Data  Link  provides  digital,  two- 
way  communication  between  an  aircraft  and  the  ATC  system.  Unlike 
voice  radio,  Data  Link  can  offer  both  automatic  error  checking  of 
uplinks  and  downlinks,  and  a  discrete  addressing  capability  so  that 
an  aircraft  receives  only  those  communications  directed  to  it  by 
the  ground  system. 
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1.2.2  Mode  S  Data  Link. 


The  NAS  Plan  technology  which  will  be  used  to  implement  Data  Link 
is  the  Mode  Select  Beacon  System  (Mode  S) .  Mode  S  is  a  secondary 
surveillance  radar  which  will  replace  the  current  Air  Traffic 
Control  Radar  Beacon  System  (ATCRBS) .  ATCRBS  uses  a  ground-based 
rotating  antenna  that  transmits  continuous  radio  frequency 
interrogations  to  which  a  transponder  equipped  aircraft  will 
respond.  The  replies  provide  the  ATC  system  with  an  aircraft 
identification  code  or  the  aircraft  altitude.  Aircraft  location 
information  is  derived  from  the  rotational  position  of  the  antenna 
and  the  transponder  delay  at  the  time  a  reply  is  received. 

Mode  S  will  provide  the  same  functions  as  ATCRBS,  but  will  do  so 
more  accurately  while  offering  the  additional  capabilities  of  a 
Data  Link.  These  enhancements  are  possible  because  each  aircraft 
will  be  assigned  a  unique  address  code  which  will  be  used  in  the 
system's  interrogations  to  select  only  the  requested  aircraft  to 
respond.  Figure  1  illustrates  the  primary  airborne  and  ground- 
based  components  of  Mode  S . 

Because  of  its  flexibility  and  large  capacity,  the  Mode  S  Data  Link 
will  be  a  primary  enabling  technology  for  the  final  phases  of  the 
NAS  Plan  which  call  for  high  levels  of  automation  and  direct 
communication  between  ground-based  computers  and  aircraft  avionics. 
In  addition,  however,  Data  Link  holds  the  potential  for  enhancing 
the  ATC  system  in  both  the  near  term  and  the  more  distant  future 
by  providing  controllers  and  aircrew  with  a  second  means  of 
conducting  essential  communications  that  can  be  used  to  relieve  the 
growing  burden  on  the  voice  channel. 

1.2.3  Impact  on  Controllers  and  Aircrews. 

The  primary  way  in  which  Data  Link  can  improve  the  safety  and 
efficiency  of  ATC  is  through  its  potential  positive  impact  on  the 
tasks  of  the  human  operators  in  the  system.  The  first  service  to 
be  provided  by  Data  Link  will  be  automated  delivery  of  weather 
information  to  aircraft.  Pilots  will  benefit  from  this  service 
because  it  will  provide  weather  data  on  demand  which  often  will  be 
more  current  and  relevant  than  the  information  currently  available 
via  voice  radio  requests.  In  addition,  this  service  should 
favorably  affect  controller  task  load  by  eliminating  any 
requirement  to  verbally  relay  weather  data  to  aircraft,  thereby 
allowing  them  to  concentrate  on  the  primary  duty  of  maintaining 
aircraft  separation. 

Beyond  the  enhancements  that  will  be  achieved  by  removing  weather 
communications  from  the  voice  radio  channel,  even  more  significant 
improvements  will  be  possible  through  the  delivery  of  primary  ATC 
services  using  Data  Link.  ATC  instructions  such  as  transfer  of 
communication,  altitude  assignments,  and  heading  changes  currently 
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FIGURE  1.  MODE  S  SURVEILLANCE  AND  DATA  LINK 


form  the  bulk  of  radio  transmissions  which  contribute  to  frequency 
congestion.  With  a  dual  channel  communications  system  it  will  be 
possible  to  transfer  many  of  these  essential  messages  to  Data  Link, 
while  reserving  voice  radio  for  time-critical  instructions.  Thus, 
controllers  should  be  able  to  circumvent  situations  where  the  voice 
channel  is  unavailable  for  issuing  important  directions.  Likewise, 
pilots  stand  to  benefit  from  uplinked  ATC  clearances  because  they 
will  be  required  to  monitor  fewer  radio  transmissions  for  messages 
directed  to  their  aircraft.  Furthermore,  because  Data  Link  will 
provide  digital  information  to  the  avionics  display  system,  pilots 
will  have  an  accurate  and  recallable  record  of  uplinked  clearances. 
This  feature  should  reduce  the  pilot's  memory  load  and  reduce 
misinterpretation  of  messages,  requests  for  repeats  of 
instructions,  and  possible  flight  control  errors. 

A  final  potential  advantage  of  Data  Link  for  both  the  airborne  and 
ground-based  participants  in  the  ATC  system  will  accrue  from  its 
application  as  a  general  secondary  communications  channel.  The 
availability  of  Data  Link  for  emergency  communications  required 
because  of  radio  failures,  mistuned  radios,  and  blocking  of 
channels  by  inadvertently  actuated  microphone  switches  will  provide 
both  pilots  and  controllers  with  a  "safety  net"  that  is  unavailable 
in  the  current  system. 

1.2.4  Requirement  for  Operational  Testing  of  Data  Link. 

As  noted  above,  Data  Link  offers  a  number  of  technical  features 
which  have  the  capability  to  enhance  the  safety  and  productivity 
of  the  ATC  system.  However,  Data  Link  will  also  profoundly  change 
both  the  way  in  which  ATC  tasks  are  accomplished  by  controllers  and 
the  way  in  which  pilots  will  receive  and  respond  to  ATC 
instructions.  Because  of  this,  the  ultimate  success  of  Data  Link 
is  critically  dependent  on  the  extent  to  which  it  is  employed  to 
produce  a  system  that  is  fully  integrated  with  the  human  operators 
who  will  be  required  to  use  it. 

The  goal  of  the  operational  evaluation  study  reported  here,  and 
that  of  other  pilot  and  controller  studies  conducted  under  the  Data 
Link  program,  is  to  promote  this  human-machine  integration  through 
scientific  testing  of  procedures  and  pilot-controller  interfaces 
under  simulated  ATC  and  flight  conditions.  The  specific  intent  of 
this  program  of  research  and  evaluation  is  to  assess  the  actual 
benefit  of  Data  Link  to  ATC  and  to  insure  that  these  benefits  are 
not  outweighed  by  decrements  in  the  performance  or  increases  in  the 
workload  of  controllers  and  aircrew  who  use  the  system. 

1.3  DATA  LINK  TEST  BED. 

The  research  reported  in  this  document  was  conducted  in  the  Data 
Link  Test  Bed  Facility  located  at  the  FAA  Technical  Center.  The 
test  bed  was  assembled  to  provide  both  engineering  testing  and 
evaluation  capabilities  during  controller  and  pilot  in-the-loop 


simulation.  Figure  2  presents  a  functional  diagram  of  the  main 
components  of  the  test  bed  and  their  interconnections. 

A  central  feature  of  the  test  bed  facility  is  th^.f  it  n^r-mitc; 
development  and  evaluation  of  Data  Link  services  on  actual  en  route 
ATC  workstations  and  associated  computers  rather  than  on  emulation 
systems.  As  a  result,  subject  controllers  are  able  to  make 
confident  evaluations  of  the  operational  feasibility  of  the  tested 
services.  In  addition,  the  capability  to  integrate  high  fidelity 
flight  simulators  into  the  communications  and  radar  display  systems 
of  the  test  bed  makes  it  possible  to  elicit  similarly  confident 
judgments  from  pilots,  and  to  examine  issues  involving  the 
interaction  between  controllers  and  pilots. 

During  each  test  run  of  the  simulation  study  described  in  this 
report,  subject  controllers  were  presented  with  simulated  radar 
data  and  ATC  system  information  from  the  HOST  computer.  The 
majority  of  the  simulated  aircraft  in  the  test  scenarios  were 
controlled  from  computer  terminals  by  simulation  operators  who  also 
provided  voice  and  Data  Link  responses  to  the  air  traffic 
controllers . 

In  addition,  two  of  the  aircraft  in  each  scenario  were  full 
fidelity  flight  simulators  linked  to  the  test  bed  and  under  the 
control  of  professional  and  private  pilots  serving  as  test 
subjects.  One  of  these  aircraft  was  a  Boeing  727  simulator  located 
at  the  National  Aeronautics  and  Space  Administration  (NASA)  Ames 
Research  Center  in  California  and  manned  by  commercial  airline 
flight  crews.  The  second  was  a  GAT-2  general  aviation  simulator 
located  at  the  FAA  Technical  Center  and  controlled  by  private 
pilots . 

1.4  TESTED  DATA  LINK  SERVICES. 


The  initial  package  of  services  scheduled  for  implementation  in  the 
Data  Link  system  and  tested  in  this  study  includes  weather 
information  and  four  ATC  services  and  functions.  These  ATC 
services  and  functions  are  altitude  assignment,  transfer  of 
communication,  menu  text,  and  free  text. 

The  altitude  assignment  service  will  transmit  digital  altitude 
clearances  to  an  aircraft  display  as  a  substitute  for  the  voice 
radio  clearance  used  in  the  current  system.  Transfer  of 
communication  is  presently  accomplished  by  a  voice  contact  with  a 
pilot  following  a  transfer  of  control  between  two  sectors.  Using 
Data  Link,  a  digital  message  containing  the  new  radio  frequency 
will  be  prepared  automatically  when  the  transfer  of  control  is 
accepted.  The  transferring  controller  will  then  uplink  the  new 
frequency  to  the  aircraft  before  it  crosses  the  sector  boundary. 
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data  link  test  bed 


Menu  text  is  a  general  Data  Link  function  which  will  permit 
standard  or  frequently  required  ATC  messages  to  be  stored  in  menu 
form  on  the  controller's  display.  These  common  clearances  can  then 
be  uplinked  as  needed  by  designating  the  appropriate  menu  item. 
For  the  operational  evaluation,  menu  text  was  incorporated  into  the 
altitude  assignment  service  to  provide  uplinks  of  repetitively 
required  interim  altitudes  and  fix  crossing  restrictions. 

Free  text  will  provide  controllers  with  a  capability  to  engage  in 
unconstrained  ground-air  communications  with  Data  Link  equipped 
aircraft  using  typed  keyboard  entries.  This  service  will  permit 
controllers  to  compose  and  uplink  textual  messages,  thereby 
providing  an  emergency  back-up  channel  for  voice  transmissions. 

In  the  present  study,  controllers  used  the  four  ATC  services  and 
functions  described  above  to  provide  pilots  with  radio  frequency 
changes,  unconstrained  free  text  messages,  and  altitude  clearances 
which,  when  sent  via  menu  text,  could  include  crossing 
restrictions.  The  aircraft  Data  Link  displays  were  touch- 
sensitive,  light  emitting  diode  (LED)  units  which  presented 
clearance  and  weather  information  and  received  manual  response 
inputs  for  downlink  to  the  controllers.  Additional  pilot  inputs 
to  the  unit  were  used  for  display  management  and  issuing  weather 
requests.  As  intended  in  the  Data  Link  system  design,  the  weather 
services  involved  direct  interaction  between  the  pilots  and  a 
weather  data  base.  Since  the  controllers  did  not  participate  in 
these  transactions,  they  did  not  evaluate  the  weather  services. 

1.5  ORGANIZATION  OF  THE  REPORT. 


The  following  sections  of  this  report  present  the  detailed  results, 
conclusions,  and  recommendations  derived  from  the  operational 
evaluation  study.  Because  somewhat  different  issues  were  addressed 
from  the  two  perspectives,  individual  chapters  are  devoted  to  the 
design  and  results  of  the  controller  (section  2)  and  aircrew 
(section  3)  substudies.  Section  4  emphasizes  those  results  of  both 
substudies  which  bear  on  joint  aircrew  and  controller  issues. 
Section  5  presents  the  overall  conclusions  of  both  substudies,  and 
section  6  lists  the  recommendations  for  future  work  that  were 
derived  from  the  operational  evaluation. 

All  appendixes  referenced  in  sections  2  through  4  are  contained  in 
volume  II  of  this  report. 

2.  DATA  LINK  CONTROLLER  SUBSTUDY. 

L  OBJECTIVES . 

L.l  The  Data  Link  Controller  Evaluation  Program. 

The  overall  test  plan  for  controller  evaluation  of  Data  Link  ATC 
services  calls  for  research  to  achieve  two  major  objectives. 
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First,  the  results  of  controller  studies  must  determine  the  design 
of  operational  procedures,  data  display  methods,  and  controller 
inputs  for  each  of  the  subject  services.  Second,  these  studies 
must  also  evaluate  the  impact  of  Data  Link  services  on  overall 
operational  ATC  system  performance  and  insure  that  they  are 
acceptable  to  controllers  and  fully  usable  in  operational 
environments . 

Iii  order  to  address  both  of  these  objectives  in  an  efficient 
manner,  a  two-stage  process  was  devised  to  evaluate  individual 
groups  of  Data  Link  services  as  they  are  developed.  In  the  first 
stage,  highly  controlled,  part-task  simulation  studies  are 
conducted  to  reduce  the  number  of  potential  service  design 
alternatives  to  a  reasonable  set.  These  "mini"  studies  focus  on 
formal  testing  of  display  and  procedural  options  rather  than  on 
high  fidelity  simulation  of  the  ATC  environment.  The  primary 
criteria  for  option  evaluation  in  the  mini  studies  are  measures  of 
projected  controller  workload  and  acceptance.  These  metrics  are 
supplemented  by  structured  discussions  and  design  critique  sessions 
conducted  to  elicit  expert  controller  opinions. 

In  the  second  stage  of  the  evaluation  process,  operational 
evaluation  studies  are  planned  to  exercise  the  finalized  designs 
in  the  context  of  full-scale  ATC  simulations.  The  general  goals 
of  the  operational  evaluation  stage  are  to  confirm  the  optimality 
of  the  design  for  each  Data  Link  service,  and  to  verify  the 
projected  benefits  of  Data  Link  under  various  realistic  operational 
conditions . 

2.1.2  Preliminary  Controller  Mini  Studies. 

In  accordance  with  this  test  plan,  two  mini  design  studies  and 
several  design  generation  and  critique  sessions  were  conducted  on 
the  initial  package  of  Data  Link  services  with  en  route  air  traffic 
controllers  from  the  Dallas/Ft.  Worth  Air  Route  Traffic  Control 
Center  ( ARTCC) ,  and  with  the  recently  formed  Air  Traffic  Data  Link 
Validation  Team  ( ATDLVT) .  These  iterative  research  efforts 
examined  basic  display  and  procedural  issues  and  produced  designs 
for  the  transfer  of  communication,  altitude  assignment,  menu  text, 
and  free  text  services. 

Complete  documentation  of  the  results  of  these  mini  studies  are 
contained  in  two  FAA  Technical  Center  reports  (Talotta,  et  al., 
1988,  1989).  Particularly  noteworthy  findings  of  the  studies 
included  strongly  positive  controller  responses  to  Data  Link  and 
th.p  individual  services  that  were  evaluated.  Specific  results 
produced  modified  display  designs  which  present  the  status  of 
ongoing  Data  Link  transactions  in  both  a  list  form  on  the  Plan  View 
Display  ( PVD) ,  and  in  an  abbreviated  form  in  the  Full  Data  Block 
( FDB) .  Controller  inputs  from  the  studies  also  contributed  to  the 
evolution  of  service  designs  which  minimize  keyboard  and  trackball 
entries  with  automatic  preparation  and  deletion  of  messages. 
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Procedurally ,  the  controllers  who  participated  in  the  mini  studies 
preferred  service  delivery  methods  which  were  designed  to  be  as 
similar  as  possible  to  current  ATC  practices.  In  addition,  the 
controllers  supported  the  use  of  Data  Link  as  a  primary  service 
delivery  mode  rather  than  as  a  confirmatory  follow-up  to  voice 
clearances . 

As  dictated  by  the  results,  all  designs  require  a  downlinked 
"WILCO"  response  from  pilots  to  complete  a  transaction.  Finally, 
while  automatic  procedures  were  considered  useful  by  the  test 
subjects,  the  findings  of  the  mini  studies  also  revealed  a 
requirement  to  design  a  majority  of  the  services  with  a  capability 
for  controllers  to  maintain  manual  control  over  the  specific 
destination  for  entries  into  the  system  (e.g.,  to  the  pilot,  to  the 
NAS  data  base,  to  the  FDB  display,  or  any  combination  of  the  three) 
and  over  the  time  at  which  an  uplink  occurs. 

2.1.3  Objective  of  the  Operational  Evaluation. 

The  research  reported  in  this  document  represents  the  culmination 
of  the  series  of  studies  conducted  on  the  initial  package  of  Data 
Link  ATC  services  and  functions.  As  mandated  by  the  test  plan, 
this  study  was  designed  to  assess  the  optimality  of  the  services 
developed  during  the  mini  studies  in  the  context  of  a  high  fidelity 
simulation  of  en  route  ATC  activity.  The  specific  objectives  of 
the  study  were  to  assess  the  projected  beneficial  impact  of  Data 
Link  on  voice  frequency  congestion,  and  to  determine  the  effects 
of  the  introduction  of  the  initial  package  of  Data  Link  services 
on  operational  ATC  system  performance,  controller  workload,  and 
controller  acceptance.  The  operational  tests  described  in 
succeeding  portions  of  this  section  of  the  report  evaluated  each 
of  these  factors  in  two  realistic  test  scenarios,  and  under 
aircraft  Data  Link  equipage  levels  representing  both  early  and  late 
stages  of  system  implementation. 

2.2  TEST  APPROACH. 


The  approach  that  was  adopted  to  meet  the  objectives  of  this 
operational  evaluation  involved  the  participation  of  eight  en  route 
air  traffic  controllers  in  a  series  of  30-minute  ATC  test  scenarios 
presented  at  the  workstations  in  the  Data  Link  test  bed.  Each  test 
run  required  a  subgroup  of  four  of  the  controllers  to  actively 
control  traffic  in  adjacent  sectors  of  en  route  airspace.  The  two 
test  scenarios  provided  realistic  ATC  problems  involving  the 
movement  of  traffic  between  two  airports,  climbing  and  descending 
aircraft,  departing  and  arriving  aircraft,  handling  interfering 
overflight  traffic,  and  merging  of  different  streams  of  traffic 
into  a  single  air  route. 

During  each  test  run,  objective  data  were  automatically  collected 
on  controller  usage  of  voice  frequencies  and  Data  Link.  Following 
each  test,  the  subjects  provided  quantitative  workload  ratings  and 
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reports  of  any  incidents  that  may  have  affected  the  safety  or 
efficiency  of  ATC  during  the  run.  In  addition,  after  completing 
all  test  runs,  the  subjects  rated  both  the  operational 
effectiveness  and  acceptability  of  each  of  the  four  service 
designs.  Post-test  debriefing  sessions  were  used  to  elaborate  and 
explain  the  results  obtained  in  the  test  bed  and  to  complete  a 
wrap-up  questionnaire. 

The  rationale  underlying  the  test  design  was  to  create  a  series  of 
realistic  ATC  test  conditions  in  which  it  would  be  possible  to 
evaluate  the  predicted  benefits  of  the  Data  Link  services  in  terms 
of  their  effect  on  voice  frequency  usage  in  comparison  to  current 
voice-only  conditions.  Measures  of  controller  workload,  system 
effectiveness,  and  user  acceptance  were  employed  to  determine 
whether  any  measured  benefits  of  Data  Link  would  be  potentially 
outweighed  by  corresponding  "costs"  to  ATC  system  performance. 

2.3  TEST  CONDUCT. 

2.3.1  Subjects. 

The  subjects  for  this  study  were  eight  en  route  air  traffic 
controllers  from  the  Anchorage,  Cleveland,  Memphis,  Miami, 
Dallas/Ft.  Worth,  and  Washington  ARTCC's.  All  but  two  of  the 
subjects  were  current,  full  performance  level  (FPL)  controllers. 
The  FPL  subjects  had  an  average  of  11.2  years  of  experience  with 
a  range  of  2.5  to  33  years. 

Two  of  the  subjects  had  participated  in  earlier  Data  Link  mini 
study  evaluations.  The  remaining  six  subjects  had  not  received  any 
formal  exposure  to  the  specific  Data  Link  service  designs  or 
procedures  tested  in  the  present  study  prior  to  their  participation 
in  the  operational  evaluation. 

2.3.2  Test  Configurations  for  Data  Link  Services. 

As  noted  earlier  in  this  section  of  the  report,  the  displays, 
controller  inputs,  and  procedures  for  the  four  Data  Link  services 
tested  in  this  study  were  based  on  the  findings  of  preliminary  mini 
studies  and  design  generation  meetings  conducted  with  controllers 
at  the  FAA  Technical  Center.  In  accordance  with  those  results,  The 
Data  Link  transaction  list  was  located  on  the  PVD  rather  than  a 
peripheral  display.  In  addition,  for  transfer  of  communication  and 
altitude  assignment,  an  FDB  display  of  key  transaction  states  was 
provided.  The  FDB  also  contained  a  Data  Link  equipage  symbol  which 
identified  those  aircraft  capable  of  receiving  uplinks,  and 
indicated  whether  a  controller  was  eligible  to  uplink  to  the 
aircraft.  Eligibility  for  both  Data  Link  and  voice  communications 
was  transferred  to  a  receiving  controller  after  a  transfer  of 
communication  uplink  had  been  received  from  a  sending  controller 
and  acknowledged  by  a  pilot  via  a  downlink. 
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Altitude  assignments  were  accomplished  by  entering  the  information 
to  the  NAS  data  base  in  the  same  manner  as  this  system  update  is 
currently  accomplished.  If  the  controller  chose  to  uplink  the 
assignment,  the  entry  was  preceded  by  an  "S"  to  initiate  the 
transaction.  For  this  study,  the  implementation  of  menu  text  was 
used  only  in  conjunction  with  temporary  altitude  assignments. 
Fixed  menus  containing  appropriate  temporary  altitudes  and  crossing 
restrictions  were  prepared  prior  to  the  study  and  displayed  on  the 
PVD .  Controllers  selected  specific  messages  using  the  trackball 
or  by  typing  the  message's  letter  code.  Free  text  messages  were 
composed  on  the  controller's  keyboard  in  real  time  and  uplinked  by 
a  single  key  stroke. 

Detailed  descriptions  of  the  procedures  and  display  formats  for 
each  of  the  Data  Link  ATC  services  outlined  above  are  documented 
in  appendix  A. 

2.3.3  Test  Scenarios. 

Three  basic  scenarios  were  created  for  the  operational  evaluation 
study.  One  of  these  was  a  training  scenario  used  only  during 
initial  familiarization  sessions  designed  to  teach  the  subjects  the 
procedures  for  composing  and  transmitting  Data  Link  messages.  The 
remaining  two  scenarios  were  test  scenarios  used  during  the  data 
collection  runs.  All  scenarios  used  the  Universal  Data  Set  (UDS) 
adaptation  of  the  en  route  HOST  system.  UDS  is  a  fictional 
airspace  which  includes  a  variety  of  air  routes  and  geographical 
features.  UDS  was  chosen  in  lieu  of  an  actual  site  adaptation  to 
avoid  any  effects  of  preliminary  differences  among  the  subjects  in 
familiarity  with  the  test  air  space.  Fixes  and  sector  boundaries 
were  selected  from  UDS  to  create  the  air  routes  used  in  the  three 
scenarios.  Appendix  A  presents  detailed  documentation  and  maps  for 
all  versions  of  the  three  basic  scenarios  employed  in  the  study. 
A  brief  description  of  the  scenarios  is  presented  below. 

The  training  scenarios  contained  two  routes  carrying  traffic 
between  airports  arbitrarily  defined  as  Philadelphia  and  Atlantic 
City.  One  of  the  routes  was  used  for  outbound  traffic  from 
Philadelphia  which  departed  through  a  low  altitude  sector,  crossed 
two  high  altitude  sectors,  and  descended  to  a  low  altitude  sector 
to  arrive  at  Atlantic  city.  The  second  route  completed  the  loop 
between  the  two  airports,  traversing  the  same  sectors  in  the 
opposite  direction.  Two  versions  of  the  training  scenario  were 
presented  to  the  subjects.  In  one  of  these,  all  24  aircraft  were 
Data  Link  equipped.  In  the  second  version,  only  70  percent  of  the 
45  aircraft  were  Data  Link  equipped.  Furthermore,  in  this  version 
of  the  scenario  the  additional  21  aircraft  were  traveling  on  two 
overflight  routes  designed  to  conflict  with  the  inter-airport 
traffic. 

Test  scenario  1  was  topologically  similar  to  the  training  scenario, 
presenting  a  circular  routing  between  two  airports.  However,  both 
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the  complexity  of  the  routes  and  the  traffic  volume  were  increased. 
Twenty-five  of  the  aircraft  traveled  the  routes  between  the 
airports,  while  the  remaining  39  were  overflights  traveling  four 
different  routes  which  conflicted  with  the  primary  flow. 

Test  scenario  2  presented  two  separate  flows  of  air  traffic  which 
operated  independently  of  one  other.  The  first  flow  simulated  two 
approximately  parallel  departure  routes  from  an  airport  through  a 
low  altitude  sector  and  into  a  high  altitude  sector.  A  total  of 
45  aircraft  interacted  in  this  traffic  flow  with  12  traveling  over 
each  departure  route  and  21  conflicting  with  the  flow  on  three 
intersecting  overflight  routes.  The  second  of  the  two  traffic 
flows  represented  an  arrival  problem  in  which  two  streams  of 
traffic  were  to  be  merged  over  a  fix  in  a  high  altitude  sector  and 
descended  in  a  single  stream  through  a  low  altitude  sector.  A 
total  of  33  aircraft  interacted  in  this  traffic  flow  with  26 
traveling  over  the  arrival  routes  and  7  over  a  conflicting 
overflight  route  which  passed  through  the  low  altitude  sector. 

Both  of  the  test  scenarios  were  used  in  conditions  in  which  all 
communications  were  conducted  with  voice  radio,  and  in  which  20  and 
70  percent  of  the  aircraft  were  Data  Link  equipped.  In  addition 
to  the  simulated  aircraft  tracks  described  above,  both  of  the 
manned  aircraft  simulators  used  in  the  aircrew  substudy 
participated  in  the  test  scenarios.  Section  3  of  this  report 
includes  a  description  of  the  flightpaths  traversed  by  these 
aircraft. 

2.3.4  Test  Procedures . 

The  operational  evaluation  study  was  conducted  over  a  period  of  5 
days.  The  first  day  was  devoted  to  subject  prebriefing  and 
training.  Days  2  and  3  were  used  for  simulation  runs  in  the  Data 
Link  test  bed.  Additional  data  were  collected  during  the 
debriefing  sessions  which  occurred  on  the  final  2  days  of  the 
study. 

The  prebriefing  session  began  with  a  series  of  short  presentations 
designed  to  acquaint  the  subject  controllers  with  the  general 
concept  of  Data  Link,  the  overall  Data  Link  test  plan,  and  the  four 
ATC  services  under  evaluation.  All  presentations  regarding  the  ATC 
services  were  conducted  by  a  subgroup  of  four  controllers  from  the 
ATDLVT  who  had  participated  in  the  design  of  the  Data  Link  services 
and  in  the  development  of  the  test  scenarios. 

Classroom  training  on  the  procedures  used  to  deliver  each  of  the 
services  was  followed  by  a  briefing  on  the  measures  that  were  to 
be  used  during  the  test  sessions.  This  briefing  explained  all 
rating  scales  that  the  controllers  would  be  asked  to  complete,  and 
discussed  the  measures  of  voice  radio  and  Data  Link  activity  that 
would  be  collected  automatically  by  the  simulation  system. 


The  final  4  hours  of  Day  1  were  devoted  to  training  in  the  Data 
Link  test  bed.  rach  subject  received  approximately  2  hours  of 
hands-on  experience  with  the  detailed  Data  Link  procedures  that  had 
been  described  during  the  prebriefings.  During  the  training 
session  the  subjects  controlled  air  traffic  in  the  training 
scenarios  under  the  guidance  of  ATDLVT  members  who  were  thoroughly 
familiar  with  the  system.  Training  was  terminated  when  the  ATDLVT 
members  judged  that  the  subjects  had  achieved  a  sufficient  level 
of  expertise  to  control  air  traffic  in  the  two  test  scenarios.  It 
should  be  noted  that  in  order  to  avoid  over  exposure  to  the  test 
scenarios,  the  subjects  did  not  receive  explicit  training  with 
these  scenarios  during  the  training  session.  Thus,  other  than  the 
opportunity  to  inspect  route  maps  prior  to  the  start  of  testing, 
the  subjects'  first  experience  with  the  test  scenarios  occurred 
during  actual  data  collection. 

Data  collection  in  the  test  bed  occurred  on  Days  2  and  3  with  one 
subgroup  of  four  of  the  subjects  participating  as  controllers  on 
each  day.  Each  subgroup  completed  seven  data  collection  runs.  The 
first  four  runs  employed  test  scenario  1  while  the  last  three  runs 
were  conducted  with  test  scenario  2.  The  primary  independent 
variable  which  differentiated  the  test  runs  was  the  level  of  Data 
Link  equipage  for  the  aircraft  simulated  in  the  test  scenarios. 
One  run  under  each  scenario  was  conducted  under  current  conditions 
with  no  Data  Link  equipped  aircraft.  This  test  was  included  as  a 
comparative  baseline  against  which  the  Data  Link  runs  could  be 
evaluated.  The  remaining  runs  for  each  scenario  presented  two 
levels  of  partial  equipage  in  the  simulated  aircraft  population. 
In  the  low  equipage  condition,  20  percent  of  the  aircraft  were 
designated  as  Data  Link  equipped  to  produce  an  ATC  environment 
similar  to  that  expected  during  relatively  early  stages  of 
implementation  of  the  Data  Link  system.  In  the  high  equipage 
condition,  70  percent  of  the  aircraft  in  each  scenario  had  Data 
Link  capability  in  order  to  simulate  a  more  advanced  stage  of 
system  implementation. 

For  all  partial  equipage  test  runs,  Data  Link  equipped  aircraft 
were  identified  by  an  equipage  symbol  located  in  the  first  position 
of  the  first  line  in  the  corresponding  FDB  (see  appendix  A) .  The 
test  controllers  were  instructed  to  use  Data  Link  to  communicate 
with  these  aircraft  whenever  appropriate  and  possible,  but  to  use 
their  personal  judgement  in  switching  to  voice  transmissions  if 
necessary . 

In  addition  to  the  primary  comparison  of  two  levels  of  Data  Link 
equipage  to  the  baseline,  voice-only  condition,  the  test  sessions 
included  an  evaluation  of  the  potential  impact  of  Data  Link 
transmission  delay.  For  both  subgroups  of  subjects,  the  fourth 
run  under  test  scenario  1  was  a  second  test  of  the  70  percent  level 
of  Data  Link  equipage.  It  differed  from  the  first  presentation  of 
this  condition  only  in  the  average  time  delay  between  the 
initiation  of  an  uplink  to  its  receipt  by  the  airborne  system,  and 
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from  the  initiation  of  a  pilot  downlink  to  its  receipt  by  the 
ground  system. 

Because  the  Mode  S  system  is  a  narrow  beam  radar,  these  technical 
delay  times  will  be  determined  by  the  rotation  speed  of  the 
antenna.  While  all  other  test  runs  were  conducted  using  a  6- 
second  assumed  rotation  period,  the  extended  delay  run  doubled  this 
time  to  12  seconds.  Adding  a  1-second  computer  processing  time, 
these  parameters  yielded  an  average  uplink  delay  of  4  seconds 
(ranging  from  1  to  7  seconds)  for  the  normal  conditions,  and  an 
average  uplink  delay  of  7  seconds  (ranging  from  1  to  13  seconds) 
for  the  "worst  case"  extended  delay  condition. 

For  downlinks,  Mode  S  delays  will  be  partially  dependent  on  the 
latency  of  the  pilot's  response  to  an  uplink.  If  this  response 
time  is  less  than  the  time  taken  for  a  single  scan,  then  the 
average  downlink  delay  will  be  approximately  equal  to  the  scan 
period.  However,  if  pilot  responses  are  longer,  downlink  delays 
will  increase  in  even  multiples  of  the  scan  period.  Thus,  for  the 
present  study,  pilot  responses  which  occurred  in  less  than  6 
seconds  after  message  receipt  were  received  by  controllers  in  6 
seconds  in  the  normal  delay  conditions.  However,  a  similar 
response  latency  in  the  extended  delay  condition  resulted  in  a  12- 
second  wait  for  receipt  of  the  downlink. 

In  all  Data  Link  test  runs,  the  simulation  system  was  designed  so 
that  5  percent  of  uplinks  attempted  by  the  controllers  would  not 
be  completed  due  to  a  technical  failure.  This  outcome  required  the 
subjects  either  to  resend  the  message  or  contact  the  pilot  by  voice 
radio,  at  their  discretion.  Although  the  actual  rate  of  technical 
acknowledgement  failure  is  predicted  to  be  much  lower  in  the 
Mode  S  system,  the  5  percent  level  was  used  in  this  study  to 
increase  the  subjects'  experience  with  potential  uplink  problems. 
In  addition  to  technical  failures,  if  a  simulation  operator  acting 
as  a  pilot,  or  either  of  the  two  aircraft  simulator  pilots  failed 
to  respond  to  an  uplink  within  40  seconds  after  it  had  been 
technically  acknowledged  by  the  simulated  airborne  processor,  the 
controller  received  a  "FAIL"  message.  This  outcome  required  the 
same  controller  response  as  a  failed  technical  acknowledgement. 


The  order  of  presentation  that  was  used  for  the  seven  test  runs  in 
each  of  the  two  controller  subgroups  is  shown  below: 


Subgroup  1 


Order 


Scenario  Data  Link  Equipage  Delay 


1 

2 

3 

4 

5 

6 
7 


1 

1 

1 

1 

2 

2 

2 


Baseline  (0) 
70% 

20% 

70% 

20% 

70% 

Baseline  (0) 


Normal 

Normal 

Normal 

Extended 

Normal 

Normal 

Normal 
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Subgroup  2 


Order 


Scenario  Data  Link  Equipage  Delay 


1 

2 

3 

4 

5 

6 
7 


1 

1 


2 

2 


20% 

70% 

Baseline  (0) 
70% 

Baseline  (0) 
70% 

20% 


Normal 

Normal 

Normal 

Extended 

Normal 

Normal 

Normal 


As  shown  above,  all  subjects  controlled  traffic  in  scenario  1  first 
and  in  scenario  2  last.  Furthermore,  the  extended  delay  run 
occurred  in  the  fourth  position  of  the  test  sequence  for  both 
subgroups.  However,  the  order  of  presentation  for  the  baseline, 
voice  radio  only  condition,  and  the  20  percent  Data  Link  condition 
was  partially  counterbalanced  across  subgroups  and  scenarios.  This 
alternation  of  baseline  runs  and  Data  Link  runs  as  the  first  and 
last  conditions  within  a  scenario  was  introduced  to  the 
experimental  design  in  order  to  allow  for  independent  statistical 
evaluation  the  effects  of  scenario  familiarity  and  Data  Link 
equipage  on  controller  workload. 

2.3.5  Data  Collection. 


Both  objective  and  quantified  subjective  measures  were  collected 
during  the  test  bed  simulation  sessions.  In  order  to  assess  the 
impact  of  introducing  Data  Link  to  the  ATC  environment,  and  to 
estimate  its  affect  on  frequency  congestion,  the  simulation  system 
computers  automatically  recorded  data  on  voice  radio  and  Data  Link 
usage.  For  each  simulation  run,  the  test  bed  voice  communications 
system  detected  and  recorded  all  microphone  switch  activations  and 
deactivations  initiated  by  the  subject  controllers  on  the  aircraft 
communications  channels.  Analysis  software  applied  to  the 
recordings  tallied  the  number  of  activation-deactivation  events  in 
each  test  condition  as  well  as  the  duration  of  each  of  these 
events . 

Similar  recordings  of  Data  Link  transactions  provided  indices  of 
the  number  and  types  of  services  uplinked  during  each  test  run. 
As  noted  earlier,  a  majority  of  these  transactions  were  with 
simulation  operators  acting  as  pilots  in  the  laboratory.  Because 
these  individuals  were  not  qualified  aircrew  and  were  not  engaged 
in  realistic  flight  tasks,  their  latencies  for  WILCO  downlinks 
were  not  valid  estimates  of  the  response  times  for  pilots.  Thus, 
no  recordings  were  made  of  the  time  required  by  the  simulation 
operators  to  complete  the  delivery  of  a  Data  Link  service. 
However,  this  parameter  was  assessed  for  the  transactions  with  the 
aircrew  who  flew  the  flight  simulators  in  each  test  run.  The 
methods  used  for  collecting  these  data  and  the  results  that  were 
obtained  are  described  in  the  section  3  of  this  document  (Aircrew 
Substudy) . 


15 


A  break  of  approximately  15  minutes  was  scheduled  at  the  end  of 
each  simulation  run  in  order  to  permit  each  subject  to  rate  the 
level  of  workload  experienced  during  the  prior  period  and  to  record 
any  comments  regarding  the  test.  Controller  workload  was  measured 
using  the  Subjective  Workload  Assessment  Technique  (SWAT) .  SWAT 
is  a  workload  rating  and  scaling  system  which  was  developed  in  the 
early  1980's  by  the  U.S.  Air  Force  as  a  standardized  method  for 
obtaining  quantified  estimates  of  perceived  workload  in  a  broad 
variety  of  occupational  tasks.  SWAT  has  received  extensive  use  in 
simulation  and  operational  testing  environments  within  the 
Department  of  Defense,  and  was  used  successfully  with  air  traffic 
controllers  during  the  preliminary  Data  Link  mini  studies. 

Briefly,  SWAT  consists  of  a  set  of  three  3-point  rating  scales 
referring  to  the  dimensions  of  time  load,  mental  effort,  and 
psychological  stress.  Subjects  indicate  the  workload  of  a  session 
of  activity  by  marking  the  appropriate  points  on  each  scale.  A 
unique  feature  of  SWAT  is  that  these  ordinal  ratings  are  converted 
to  single  points  on  an  interval  measurement  scale  ranging  from  0 
(low  workload)  to  100  (high  workload)  using  a  mathematical  analysis 
method  known  as  conjoint  measurement.  This  method  not  only  yields 
data  which  are  amenable  to  powerful,  parametric  statistical 
testing,  but  also  tailors  the  measurement  scale  to  each 
individual's  (or  homogeneous  group's)  concept  of  how  the  time, 
effort,  and  stress  dimensions  combine  to  produce  an  overall 
perception  of  workload. 

The  interval  measurement  scale  used  to  interpret  the  ordinal 
ratings  is  created  by  having  subjects  generate  an  ordering  of  all 
27  combinations  of  the  time,  effort,  and  stress  levels  which 
reflects  their  individual  concept  of  how  the  three  dimensions 
combine  to  produce  different  workload  levels.  The  card  sorting 
task  used  during  this  scale  development  exercise  was  completed  by 
all  eight  controller  subjects  during  the  prebriefing  session  on 
the  first  day  of  the  study.  A  copy  of  the  SWAT  scale  used  by  the 
subjects  to  make  their  workload  ratings  and  a  transcript  of  the 
instructions  for  the  card  sort  task  are  included  in  appendix  B. 

After  completing  the  SWAT  rating  for  the  previous  test  run, 
subjects  were  given  an  opportunity  to  write  comments  on  their 
rationale  for  the  rating.  In  addition,  a  "critical  incident"  form 
was  provided  to  permit  subjects  to  record  any  ATC  events  that  had 
occurred  during  the  prior  test  run  which  would  have  compromised 
operational  safety  or  efficiency. 

Immediately  after  each  subject  had  completed  all  seven  test  runs 
of  the  study,  they  were  asked  to  complete  a  set  of  two  rating 
scales  for  the  four  tested  Data  Link  services.  The  first  of  these 
scales  required  a  rating  of  the  operational  effectiveness  and 
suitability  of  the  specific  design  for  a  service.  The  data  form 
permitted  the  subject  to  rate  a  service  as  "not  operationally 
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suitable"  or  on  a  7-point  scale  ranging  from  "1"  (highly  effective 
in  meeting  all  requirements  for  this  ATC  service)  to  "7"  (meets 
minimal  effectiveness  requirements) .  Instructions  to  the 
controllers  for  completing  this  scale  asked  them  to  use  their 
experience  in  the  test  bed,  as  well  as  their  prior  background  in 
operational  ATC,  to  assess  how  well  each  service  design  could 
accomplish  its  intended  task  in  the  full  range  of  en  route  field 
settings . 

The  second  rating  was  an  estimate  of  the  acceptability  and 
preferability  of  each  of  the  service  designs  to  air  traffic 
controllers.  The  data  form  permitted  the  subjects  to  rate  a 
service  design  as  "completely  unacceptable"  or  on  a  7-point  scale 
ranging  from  "1"  (highly  preferred)  to  "7"  (acceptable,  but  not 
preferred) .  Instructions  directed  the  subjects  to  consider  the 
extent  to  which  the  controller  displays,  data  input  requirements, 
and  procedures  used  to  deliver  a  service  would  be  usable  by  air 
traffic  controllers  in  field  settings. 

Additional  explicit  instructions  indicated  that  the  dimensions  of 
effectiveness  and  preference  should  be  made  independently  of  one 
another  since  a  service  might  include  all  functions  needed  to  meet 
operational  requirements  and  be  poorly  suited  to  the  way 
controllers  normally  perform  their  duties.  Likewise,  a  design 
could  be  easy  to  use  but  have  missing  functions  which  prevent  it 
from  meeting  operational  needs. 

Days  4  and  5  of  the  study  were  devoted  to  debriefing  sessions.  All 
eight  controllers  participated  in  these  group  sessions  in  a 
conference  room  setting  with  test  personnel  in  attendance.  The 
subjects  first  completed  a  wrap-up  questionnaire  containing  items 
requiring  numerical,  ordinal  level  ratings,  and/or  written 
comments.  The  questionnaire  addressed  specific  details  of  the 
designs  for  the  four  Data  Link  services,  the  general  impact  of  Data 
Link  on  ATC,  and  the  subjects'  impressions  of  the  quality  of  the 
operational  evaluation  study. 

After  completing  the  individual  wrap-up  questionnaires,  the 
subjects  participated  in  a  structured  discussion  session  designed 
to  elicit  final  comments  on  the  Data  Link  system.  This  session  was 
guided  by  test  personnel  using  issues  and  questions  presented  on 
overhead  projection  slides.  Hard  copies  of  the  slides  were 
provided  to  the  subjects  to  record  individual  comments.  The 
content  of  the  verbal  discussions  was  recorded  in  test  personnel 
notes  and  on  audio  tape  for  later  review. 

2 . 4  TEST  RESULTS . 


The  detailed  results  obtained  with  the  data  collection  methods 
described  in  section  2.3.5  are  contained  in  four  of  the  appendixes 
to  this  report.  Analyses  of  the  workload,  operational 


17 


effectiveness,  and  controller  preference  ratings  are  presented  in 
appendix  B.  Records  of  voice  and  Data  Link  transmissions  are 
contained  in  appendix  C.  Finally,  the  results  of  the  group 
debriefing  discussions  and  of  the  wrap-up  questionnaire  are 
presented  in  appendixes  D  and  E,  respectively. 

This  section  of  the  report  draws  upon  the  contents  of  all  four 
appendixes  to  address  findings  which  are  pertinent  to  each  of  the 
following  topics. 

2.4.1  Data  Link  and  Voice  Radio  Activity. 

A  primary  goal  of  Data  Link  is  to  enhance  ATC  productivity  and 
safety  by  reducing  its  dependence  on  congested  voice  radio 
frequency  channels  for  communications  between  air  traffic 
controllers  and  aircrew.  In  order  to  estimate  the  extent  to  which 
implementation  of  the  initial  package  of  four  Data  Link  services 
and  functions  will  provide  this  benefit,  detailed  records  were  made 
of  both  voice  and  Data  Link  transactions  during  all  simulation  test 
runs  in  this  study. 

For  the  baseline  runs  in  which  only  voice  radio  communications  were 
used,  controllers  completed  an  average  total  of  422  voice 
transactions.  Thus,  each  individual  controller  engaged  in  an 
average  of  approximately  105  sequences  of  microphone  activation  and 
deactivation  during  a  30-minute  test  run.  The  average  duration  of 
each  voice  sequence  was  3.8  seconds,  representing  a  total  average 
channel  occupation  time  of  1591  seconds  per  test  run.  It  should 
be  noted  that  these  times  do  not  include  the  additional 
contribution  to  frequency  congestion  of  pilot  responses  to 
controllers  or  of  pilot-initiated  communications. 

The  impact  of  introducing  Data  Link  on  these  baseline  communication 
levels  is  summarized  in  figure  3.  As  illustrated  by  this  graph, 
in  the  low  equipage  condition  where  only  20  percent  of  the  total 
aircraft  in  the  scenarios  were  eligible  for  Data  Link 
communications,  the  availability  of  the  transfer  of  communication, 
altitude  assignment,  and  text  functions  reduced  total  voice 
transmissions  by  25  percent.  This  produced  a  corresponding 
reduction  in  total  radio  channel  occupation  time  of  28  percent. 
Similarly,  in  the  high  equipage  condition  where  70  percent  of  the 
aircraft  were  eligible  for  uplinks,  the  availability  of  the  initial 
Data  Link  services  reduced  voice  transmissions  by  41  percent,  while 
radio  channel  occupation  time  dropped  45  percent.  It  should  be 
noted  that  the  25  percent  decrease  in  voice  transmissions  observed 
for  the  20  percent  Data  Link  equipage  level  may  be  an  inflated 
estimate  attributable  to  an  overall  decrease  in  the  amount  of 
communications  which  occurred  as  controllers  became  increasingly 
familiar  with  the  test  scenarios  (see  appendix  C) . 
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FIGURE  3.  DATA  LINK  AND  VOICE  MESSAGE  ISSUED  BY  CONTROLLERS 
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The  change  in  Data  Link  activity  which  accompanied  the  reduction 
in  voice  radio  usage  in  the  two  mixed  equipage  conditions  is  also 
presented  in  figure  3.  When  examining  these  data,  the  reader 
should  be  aware  that  while  voice  activity  was  measured  in  terms  of 
individual  microphone  activations,  of  which  several  may  have  been 
required  to  deliver  an  ATC  clearance,  Data  Link  activity  was 
assessed  by  tallying  complete  transactions.  Thus,  the  simple 
numbers  of  transactions  in  the  two  communications  modes  cannot  be 
directly  compared.  Nevertheless,  the  direction  and  proportional 
degree  of  change  in  the  two  indices  provide  interpretable  measures 
of  the  relative  results. 

As  shown  in  figure  3,  the  reduction  in  voice  frequency  usage  was 
generally  mirrored  by  an  increase  in  the  employment  of  Data  Link. 
However,  if  the  total  number  of  both  voice  and  Data  Link 
transmissions  in  the  mixed  equipage  conditions  are  examined,  it  can 
be  seen  that  there  was  an  overall  decrease  in  communications 
activity  in  comparison  to  the  baseline,  voice  only  tests.  This 
result  indicates  that,  in  addition  to  its  impact  on  frequency 
congestion.  Data  Link  may  provide  an  overall  increase  in  the 
efficiency  of  communication.  One  likely  source  of  this  improvement 
may  be  the  reduction  in  requirements  for  repeats  of  clearances  that 
often  are  required  when  voice  transmissions  are  misinterpreted  on 
the  radio  channel. 

Analyses  of  the  way  in  which  Data  Link  was  employed  during  the  test 
runs  showed  that  transfer  of  communication  accounted  for  29  percent 
of  the  service  deliveries,  while  altitude  assignments  accounted  for 
57  percent  of  the  transmissions.  Hard  altitudes  and  interim 
altitudes  sent  using  menu  text  accounted  for  approximately  equal 
proportions  of  the  altitude  assignment  uplinks.  Free  text  messages 
comprised  the  remaining  14  percent  of  Data  Link  transmissions. 

A  final  implication  of  these  measurements  was  derived  by  examining 
Data  Link  usage  over  time  in  the  test  scenarios.  Analyses  of  the 
density  of  air  traffic  in  the  scenarios  showed  that  the  number  of 
aircraft  handled  by  each  controller  increased  steadily  from  the 
early  to  the  later  portions  of  a  test  run.  A  similar  inspection 
of  Data  Link  activity  showed  that  uplinks  tended  to  increase  at  the 
same  rate  as  did  traffic  levels.  This  finding  indicates  that  the 
Data  Link  system  was  equally  effective  and  usable  by  controllers 
across  the  full  range  of  traffic  loads  tested  in  the  study. 

2.4.2  Controller  Workload  of  Data  Link  ATC. 

In  order  to  determine  whether  the  positive  benefit  of  Data  Link 
that  was  observed  in  this  study  had  been  achieved  at  a  cost  to  ATC 
system  capabilities  or  air  traffic  controller  performance;  measures 
of  ATC  safety  and  controller  workload  were  also  examined.  The 
quality  of  ATC  performance  obtained  in  this  study  was  assessed 
using  controller  comments  on  critical  incidents  that  were  recorded 
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after  each  test  trial.  As  noted  in  appendix  B,  these  reports 
detected  no  aircraft  separation  errors  or  other  safety  related 
events  in  any  of  the  test  runs.  Thus,  no  losses  in  ATC  capability 
were  apparent  with  Data  Link. 

While  measures  of  system  performance  provide  valuable  indices  of 
the  safety  and  efficiency  of  ATC,  they  can  be  relatively 
insensitive  to  the  long  term  effects  of  changes  to  the  system  when 
assessed  in  a  relatively  brief  series  of  simulation  runs.  This 
insensitivity  is  partially  attributable  to  the  ability  of  human 
operators  to  compensate  for  possible  system  deficiencies  in  the 
short  term  by  increasing  their  level  of  effort  expenditure. 
Because  of  this,  more  sensitive  predictive  measures  of  ATC 
capability  were  obtained  by  evaluating  the  perceived  workload  of 
the  air  traffic  controller. 

In  the  present  study,  the  SWAT  rating  and  scaling  technique  was 
used  to  compare  controller  workload  in  the  baseline,  voice  only 
conditions  and  in  the  mixed  equipage  Data  Link  test  runs.  Figure 
4  presents  the  mean  SWAT  workload  scores  produced  by  the  subjects 
in  these  conditions  for  both  of  the  test  scenarios.  Statistical 
evaluation  of  these  data  revealed  no  significant  differences 
between  the  workload  of  the  voice  radio  trials  and  either  of  the 
two  Data  Link  equipage  conditions.  As  described  in  appendix  B,  the 
single  statistically  significant  effect  detected  in  these  tests  was 
the  contribution  of  scenario  familiarity  to  controller  workload. 
That  is,  workload  tended  to  be  higher  for  initial  tests  with  a 
scenario  than  later  tests,  regardless  of  whether  Data  Link  or  full 
voice  procedures  were  used. 

The  results  of  the  workload  analysis  also  showed  no  differences 
between  the  20  and  70  percent  Data  Link  equipage  levels.  One 
hypothesis  that  had  been  considered  during  the  development  of  the 
initial  service  package  was  that  controller  workload  may  be  high 
during  the  initial  stages  of  Data  Link  implementation  when 
relatively  few  aircraft  will  be  equipped  with  Mode  S  transponders 
and  Data  Link  capabilities.  This  hypothesis  was  based  on  the 
concept  that  it  may  be  extremely  difficult  for  controllers  to  move 
from  familiar  voice  procedures  to  the  new  Data  Link  procedures  in 
order  to  service  a  small  portion  of  the  aircraft  fleet.  The 
workload  results  of  the  present  study  did  not  support  this 
hypothesis.  In  addition,  written  comments  recorded  between  test 
trials  indicated  that  the  level  of  Data  Link  equipage  had  no 
apparent  effect  on  controller  tasks,  and  that  no  problems  were 
encountered  in  shifting  between  the  two  communications  systems 
either  because  of  nonequipage  or  to  engage  in  expedient 
communications  with  Data  Link  equipped  aircraft. 

2.4.3  Data  Link  Transmission  Delays. 


The  speed  with  which  Mode  S  Link  transactions  can  occur  will  be 
partially  dependent  upon  the  rotation  period  of  the  surveillance 
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radar  antenna.  This  technical  characteristic  will  produce  a 
variable  lag  in  the  transmission  of  uplinks  or  downlinks  which  will 
range  from  nearly  instantaneous  to  several  seconds.  Since,  in 
comparison,  voice  transmissions  on  a  clear  radio  channel  are  not 
subjected  to  such  potential  delays,  an  important  issue  in 
operational  evaluation  research  is  to  determine  whether  expected 
technical  Data  Link  delays  are  tolerable  by  air  traffic 
controllers. 

In  the  present  study,  both  predicted  delays  for  a  6-second  antenna 
rotation  period  and  a  worst  case  12-second  period  were  investigated 
under  the  70  percent  level  of  aircraft  equipage.  No  differences 
in  ATC  performance  were  detected  for  the  two  delay  conditions. 
Furthermore,  SWAT  workload  ratings  revealed  no  differences  between 
the  workloads  experienced  by  the  test  controllers  in  the  two 
conditions.  Comments  recorded  after  the  extended  delay  test  trials 
and  the  results  of  the  wrap-up  questionnaire  also  indicated  that 
the  additional  delays  had  no  effect  on  the  subjects. 

2.4.4  Operational  Effectiveness  and  Controller  Preference. 

In  addition  to  the  direct  measures  of  the  impact  of  Data  Link  on 
ATC  activity  and  controller  resources  that  were  collected  in  the 
test  bed,  a  variety  of  subjective  measures  were  used  to  examine 
controller  perceptions  of  the  specific  designs  of  the  four  services 
that  were  evaluated  in  this  study. 

Two  rating  instruments  were  administered  immediately  after  the 
subjects  had  completed  the  seven  test  trials  in  order  to  obtain 
global  evaluations  of  the  operational  effectiveness  and  usability 
of  each  service.  As  shown  in  appendix  B,  none  of  the  service 
designs  were  rated  "not  operationally  suitable"  by  any  of  the  test 
subjects.  Furthermore,  the  designs  for  transfer  of  communication, 
altitude  assignment,  and  menu  text  were  all  rated  significantly 
higher  than  the  scale  midpoint,  indicating  that  they  contained 
most,  or  all  features  required  to  support  a  full  range  of 
operational  situations.  While  free  text  was  rated  lower  than  the 
other  three  services,  it  was  not  rated  significantly  poorer  than 
"meeting  most  operational  requirements." 

Ratings  of  controller  acceptance  and  preference  for  the  service 
designs  also  were  uniformly  high.  None  of  the  subjects  indicated 
that  the  procedures,  manual  inputs, or  displays  used  to  provide  any 
of  the  services  would  be  completely  unacceptable  to  field  air 
traffic  controllers.  In  addition,  the  transfer  of  communication, 
altitude  assignment,  and  menu  text  designs  were  all  rated  higher 
than  "moderately  preferred."  Free  text  was  not  rated  as  being 
significantly  different  than  this  scale  midpoint. 

The  comparatively  lower  effectiveness  and  preference  ratings 
assigned  to  free  text  appear  to  be  largely  attributable  to  specific 
experiences  with  this  service  as  reported  by  a  number  of  subjects 
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during  the  test  bed  sessions.  Although  free  text  was  designed 
primarily  for  use  in  emergency  situations  where  voice  communication 
is  lost,  some  subjects  experimented  with  using  free  text  as  a 
method  for  sending  ATC  clearances  such  as  heading  changes.  In 
doing  so,  the  subjects  noted  that  the  excessive  keyboard 
requirements  of  entering  complete  text  messages,  as  well  as  the 
potential  for  input  errors,  would  make  free  text  unsuitable  for 
issuing  clearances  under  normal  conditions.  In  light  of  these 
comments,  it  seems  likely  that  the  design  of  the  unconstrained  free 
text  service  will  be  as  effective  and  acceptable  as  the  other 
tested  services,  if  its  use  is  confined  to  the  exceptional 
situations  for  which  it  was  intended. 

2.4.5  Data  Link  Enhancements  and  Revisions. 


Although,  as  shown  above,  the  Data  Link  service  designs  were 
generally  considered  suitable  and  acceptable  for  implementation  in 
the  ATC  system,  a  number  of  suggested  enhancements  to  service 
deli  ery  procedures,  displays,  and  inputs  were  received  from  the 
test  controllers  in  written  comments  and  in  responses  to  the 
structured  debriefing  interview.  These  suggested  improvements  are 
described  in  detail  in  appendixes  B  and  D  and  are  summarized 
below. 

Individual  controllers  offered  several  comments  relevant  to  the 
design  of  Data  Link  displays.  Since  the  shift  of  some 
communications  responsibilities  from  the  auditory  to  the  visual 
channel  entails  different  controller  monitoring  demands,  these 
inputs  predictably  centered  around  potential  improvements  in  the 
ease  with  which  Data  Link  information  presented  on  the  PVD  can  be 
detected  and  interpreted.  Suggestions  included  changes  to  the 
symbology  used  to  indicate  Data  Link  equipage  and  eligibility. 
While  the  tested  diamond  and  hourglass  indicators  presented  in  the 
first  position  of  the  first  line  of  the  FDB  were  apparently 
acceptable  to  most  subjects,  some  of  the  controllers  felt  that  they 
might  be  confused  with  graphic  symbols  that  have  been  used  in  en 
route  displays  for  other  purposes.  In  addition,  one  controller 
noted  that  the  change  from  the  diamond  to  the  hourglass  symbol, 
indicating  a  controller's  eligibility  to  send  uplinks  to  an 
equipped  aircraft,  should  be  a  more  salient  event. 

Previous  mini  study  results  had  shown  that  controllers  would 
require  a  FDB  display  of  key  Data  Link  transaction  states,  as  well 
as  a  more  complete  status  list  to  reduce  the  visual  workload  of 
monitoring  ongoing  Data  Link  events.  Comments  regarding  the  FDB 
presentation  indicated  that  the  display  duration  of  6  seconds  for 
a  pilot's  downlinked  WILCO  response  may  be  too  short  to  ensure 
detection.  Several  subjects  also  noted  that  the  PVD  list  display 
became  cluttered  and  difficult  to  read  when  several  transactions 
were  in  progress  during  the  simulation  runs.  Suggestions  to 
alleviate  this  problem  included  the  possibility  of  using  a  separate 
list  for  transfer  of  communication  transactions,  or  limiting  the 
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list  content  to  only  those  transactions  which  had  failed  because 
of  technical  problems  or  the  absence  of  a  timely  pilot  response. 

The  procedural  issues  which  appeared  in  controller  comments  and 
debriefing  discussions  addressed  both  operational  procedures  for 
use  of  the  Data  Link  service  and  specific  service  design 
requirements.  As  noted  earlier,  the  subjects  concurred  that,  for 
safety  purposes,  operational  procedures  will  be  required  to 
restrict  the  use  of  free  text  to  emergency  communications  and 
other  noncritical  communications.  In  addition,  operational 
procedures  will  have  to  be  specified  to  define  the  conditions  under 
which  controllers  in  adjacent  sectors  will  be  permitted  to  issue 
Data  Link  clearances  while  a  transfer  of  communication  is  in 
progress.  With  respect  to  interactions  with  aircrew,  the  test 
controllers  agreed  that  a  standard  procedure  will  be  needed  to 
cover  the  situation  where  a  pilot  is  unable  to  comply  with  an  ATC 
instruction  issued  via  Data  Link.  Generally,  the  controllers  felt 
that  a  downlinked  UNABLE  response  was  insufficient  in  this 
contingency,  and  that  a  voice  radio  call  to  the  controller  should 
accompany,  or  be  substituted  for,  the  UNABLE  response.  The 
controllers  also  concurred  that  the  pilot's  voice  radio  check-in 
call  to  a  new  controller  could  be  eliminated  after  a  Data  Link 
transfer  of  communication. 

Finally,  the  controllers  emphasized  a  requirement  to  identify 
procedures  for  assigning  some  Data  Link  duties  to  a  radar 
associate,  "D-side",  controller.  While  the  operational  evaluation 
study  did  not  address  the  possibility  of  allocating  Data  Link  tasks 
to  other  members  of  a  control  team,  the  subjects  indicated  that  it 
would  be  feasible  for  the  radar  associate  to  provide  transfer  of 
communication  services,  and  to  compose  lengthy  free  text  messages. 

Further  specific  comments  addressed  the  detailed  procedures  used 
to  deliver  individual  Data  Link  services.  Early  mini  study 
research  had  indicated  that  while  a  transfer  of  communication 
message  could  be  generated  automatically  after  a  receiving 
controller  had  accepted  a  transfer  of  control,  the  uplink  of  the 
new  radio  frequency  should  not  occur  until  the  sending  controller 
has  made  an  explicit  input.  This  manual  uplink  procedure  was 
needed  to  make  it  possible  for  controllers  to  complete  the  transfer 
of  control  procedure  prior  to  relinquishing  voice  contact  with  an 
aircraft.  Controllers  in  the  present  study  agreed  with  this  design 
feature,  but  indicated  that  an  automatic  uplink  could  be  used  in 
some  special  situations  where  the  transfer  of  control  and  transfer 
of  communication  events  repetitively  occur  at  the  same  time.  Thus, 
it  was  suggested  that  manual  uplinks  would  be  the  normal  default 
procedure,  but  that  provision  be  made  for  the  selection  of  the 
automatic  feature. 

Minor  changes  to  the  design  of  the  menu  text  service  for  altitude 
assignments  also  were  recommended.  In  the  current  design,  separate 
keyboard  entries  are  used  to  select  "hard"  altitude  assignments  and 
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interim  altitudes.  In  the  case  of  interim  altitudes  this  input 
gives  the  controller  access  to  the  menu  of  standardized  messages 
for  simplified  uplink.  One  type  of  message  contained  in  the  menu 
automatically  uplinks  a  final  altitude  previously  requested  by  the 
pilot,  or  a  standardized  final  altitude.  Because  these  special 
menu  entries  do  not  represent  interim  altitudes,  it  was  suggested 
that  these  items  be  contained  in  a  menu  which  is  accessed  by 
pressing  the  hard  altitude  key,  rather  than  the  interim  altitude 
key. 

A  second  common  comment  regarding  the  menu  text  service  was  that 
the  items  be  extended  to  include  more  compound  clearances.  In  the 
tested  design,  the  menu  items  included  an  altitude  as  well  as  a 
geographical  fix  crossing  restriction  in  some  cases.  Since  even 
longer  compound  messages  are  commonly  sent  in  a  single  voice 
transmission  in  the  current  ATC  system,  the  controllers  suggested 
that  it  be  possible  to  expand  menu  items  to  include  such  things  as 
standard  headings. 

A  final  comment  provided  by  several  of  the  controllers  was  that 
Data  Link  should  not  be  limited  to  the  services  provided  in  the 
initial  package  tested  in  this  study.  Rather,  it  was  suggested 
that  the  system  be  rapidly  enhanced  to  provide  the  full  range  of 
ATC  services  currently  delivered  over  voice  radio  channels. 

2.4.6  Controller  Opinions  of  Data  Link  and  the  Operational 
Evaluation. 

In  agreement  with  the  findings  of  previous  mini  studies,  the 
controllers  who  participated  in  this  operational  evaluation  were 
extremely  positive  about  the  impact  that  Data  Link  will  have  on  ATC 
in  the  NAS.  Of  17  unsolicited  written  comments  on  the  overall 
effect  of  Data  Link  that  were  recorded  during  the  test  sessions, 
all  referred  to  significant  benefits  to  the  controller.  The  most 
common  remarks  included  reduced  frequency  congestion,  lowered 
requirements  to  monitor  and  respond  to  voice  read-backs  and  pilot 
check-in,  and  an  increase  in  the  accuracy  of  communication.  In 
addition,  the  results  of  the  wrap-up  questionnaire  indicated  that 
the  controllers  predicted  Data  Link  will  have  significantly 
positive  effects  on  the  overall  safety  and  efficiency  of  ATC, 
system  capacity,  and  controller  workload. 

In  evaluating  the  quality  of  the  operational  evaluation  study,  the 
realism  of  the  simulation  received  varied  ratings,  with  some 
controllers  indicating  that  the  simulation  scenarios  could  have 
been  more  varied  and  complex.  However,  despite  some  inherent 
limitations  of  the  simulation  environment,  average  controller 
ratings  of  the  traffic  in  the  two  test  scenarios  were  "moderate" 
and  only  slightly  below  the  level  where  the  subjects  indicated  that 
a  D-controller  would  have  been  required  in  an  actual  operational 
setting  to  assist  the  radar  controller.  In  addition,  the 
controllers  agreed  that  Data  Link  training  had  been  sufficient,  and 


that  they  had  received  sufficient  experience  with  each  of  the 
tested  services  to  make  a  fair  evaluation  of  the  system. 

2.5  DISCUSSION. 

The  combined  findings  of  the  ATC  portion  of  the  operational 
evaluation  study  clearly  confirmed  the  feasibility  of  implementing 
the  initial  package  of  en  route  Data  Link  ATC  services.  As  defined 
by  the  results  of  the  preliminary  mini  studies,  the  service  designs 
were  shown  to  produce  a  significant  reduction  in  voice  radio 
frequency  congestion  in  the  context  of  realistic,  manned 
simulations  of  ATC  activity.  Furthermore,  this  evaluation 
indicated  that  the  substitution  of  Data  Link  for  voice 
communication  procedures  will  not  adversely  affect  controller 
workload,  and  that  the  service  designs  would  be  both  operationally 
effective  and  acceptable  to  controllers  in  a  broad  range  of  ATC 
environments . 

In  addition  to  demonstrating  the  viability  and  potential  benefits 
of  introducing  Data  Link  to  the  contemporary  ATC  system,  the 
operational  evaluation  also  presented  an  opportunity  to  collect 
controller  inputs  regarding  additional  refinements  to  the  service 
designs.  Because  these  comments  and  suggestions  were  made  in  the 
context  of  a  previously  untested  level  of  simulation  realism,  they 
should  prove  invaluable  during  final  design  iterations.  Specific 
recommendations  derived  from  these  inputs  appear  in  section  6  of 
this  report. 

3.  DATA  LINK  AIRCREW  SUBSTUDY. 


This  section  of  the  report  documents  an  investigation  into  pilot 
perception  and  acceptance  of  the  concept  of  using  Data  Link  in 
lieu  of  voice  for  some  ATC  and  weather  (WX)  services.  As  a 
conceptual  investigation,  it  was  constrained  to  readily  available 
equipment  which  could  be  easily  adapted  to  this  purpose. 

3.1  OBJECTIVES. 

The  objective  of  the  operational  evaluation  was  to  conduct  an 
initial  (early)  exploration  of  the  concept  of  using  Data  Link  in 
lieu  of  voice  for  some  of  the  communications  between  the  aircrew 
and  ground.  Both  objective  and  subjective  measures  of 
effectiveness  were  employed.  Objective  measures  were  various 
time  intervals  associated  with  requesting,  receiving,  and 
reviewing  data  link  obtained  information.  Subjective  measures 
were  obtained  from  post-test  questionnaires. 

The  essence  of  the  Data  Link  program  is  to  change  the  way  pilots 
and  controllers  interact.  Thus,  the  major  focus  for  the 
evaluation  was  that  interaction.  The  major  question  to  be 
answered  was  whether  aircraft  operations  are  enhanced  by 
altering,  through  Data  Link,  the  way  information  is  exchanged. 
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3.2  TEST  APPROACH. 


The  aircrew  operational  evaluation  was  viewed  as  preliminary 
research  to  provide  guidance  for  follow-on  research  and  systems 
design.  The  aim  was  to  depict  pilot  performance  in  the  context 
of  the  whole  ground-air  Data  Link  system.  The  operational 
evaluation  placed  pilots  and  air  traffic  controllers  into 
realistic  interaction  by  simulating  their  tasks.  The  primary 
independent  variable  of  interest  was  the  communication  source, 
i.e..  Data  Link  vs.  voice.  A  portion  of  the  information  which  is 
currently  transmitted  by  radio  was  transmitted  through  the  Data 
Link  system. 

The  pilot's  tasks  were  to  evaluate  four  weather  (WX)  and  two  ATC 
services  that  were  exercised  under  simulated  Data  Link  and 
voice  environments.  Various  time  intervals  such  as  request 
creation  time,  system  time,  response  latency,  and  message 
processing  time  were  measured  to  assess  the  difference 
between  the  two  conditions.  Subjective  measures  in  the 
form  of  post-test  questionnaires  provided  additional  data  as 
to  the  effectiveness  of  the  Data  Link  system. 

The  pilot  tasks  were  accomplished  in  two  aircraft  simulators. 
One  of  these  was  the  General  Aviation  Trainer  (GAT) ,  which  is 
housed  at  the  FAA  Technical  Center.  The  other  was  a  simulator  of 
the  Boeing  727  (B-727)  aircraft,  which  is  located  at  the  Ames 
Research  Center  of  the  National  Aeronautics  and  Space 
Administration  (NASA) ,  Moffett  Field,  California.  The  GAT 
facility  consists  of  a  Cessna  421  motion-base  simulator.  A 
visual  system  was  not  used  in  the  evaluation.  The  B-727  at  NASA 
AMES  is  housed  in  the  Man  Vehicle  Systems  Research  Simulator 
(MVSRS)  facility.  The  system  is  a  6°  of  freedom  motion-base 
design  with  a  high-fidelity  out-the-window  visual  system. 

Each  run  was  a  flight  in  a  simulated  airspace  which  could  be 
heavily  populated  with  simulated  aircraft.  Each  pilot/crew  flew 
three  runs;  two  were  Data  Link  and  the  other  traditional  voice 
communication.  An  evaluation  team  made  up  of  human  factors 
engineers  and  pilots  brief ed/trained  the  subject  pilots  prior 
to  data  collection  exercises.  During  each  run  an  experienced 
pilot  acted  as  an  inflight  observer.  The  inflight  observer  at 
each  location  (FAA  and  NASA)  was  responsible  for  answering 
subject-pilot  questions  regarding  the  simulators,  flight  test 
scenarios,  and  any  other  pilot-specific  questions  made  by  the 
subject  pilots.  During  the  data  runs  the  inflight  observer  used 
a  cueing  sheet  to  cue  the  subject  pilots  on  weather  service 
requests  and,  as  necessary,  the  type  of  response  required  for  a 
given  service  and/or  the  appropriate  procedure  to  be  followed. 
The  inflight  observers  observed  the  data  runs  and  provided 
comments  on  the  subject  pilot's  operation  of  Data  Link. 
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Two  general  aviation  and  two  airline  oriented  flight  scenarios 
were  created  for  the  evaluation.  Various  ATC  and  weather 
services  were  exercised  in  the  data  runs.  Objective  data 
consisted  of  response  time  measures;  subjective  data  were  obtained 
in  an  extensive  questionnaire. 

3.3  TEST  CONDUCT. 

3.3.1  Subjects. 

The  pilot  subjects  were  selected  in  such  a  way  that  General 
Aviation  pilots  flew  the  GAT  and  current  airline  flight  deck 
crews  (Captain  and  First  Officer)  flew  the  727  simulator.  For 
the  FAA  GAT,  four  General  Aviation  pilots  participated,  and  for 
the  NASA  727,  four  airline  crews  participated. 

FAA  personnel  solicited  the  Aircraft  Owners  and  Pilots 
Association  (AOPA)  for  assistance  in  providing  genera^  aviation 
pilots.  Airline  type  pilots  were  obtained  through  a  h/.i A/Ames 
subject  pilot/crew  pool.  All  pilots  were  simulator  experienced. 
A  summary  of  pilot  experience  is  presented  in  table  1.  Each 
designated  pilot  subject  was  contacted  by  telephone  to  obtain 
assurance  of  availability  on  the  scheduled  days. 

An  information  packet  was  prepared  and  mailed  to  each  crew/pilot 
scheduled  to  participate.  This  packet  described  Data  Link,  the 
nature  and  purpose  of  the  test,  schedule,  and  what  was  to  be 
accomplished  on  each  day  of  the  schedule. 

3.3.2  Configurations  for  Data  Link  Services. 

3. 3. 2.1  Display  Interface. 


The  approach  to  the  Data  Link  pilot  interface  was  an  alphanumeric 
touch  sensitive  input/output  (I/O)  device  from  the  Teledyne 
Controls  Corporation.  The  Teledyne  Interactive  Display  Unit 
(IDU)  is  designed  to  display  20  characters  on  each  of  nine 
lines.  A  matrix  of  infrared  beams  is  used  to  permit  an 
interactive  capability.  The  number  of  touch  sensitive  areas  can 
be  controlled  by  programming  and  can  vary  from  one  area 
encompassing  the  whole  screen  (or  any  part)  to  as  many  as  20 
abutting  rectangular  areas.  As  implemented,  touching  a  finger  to 
one  of  the  designated  areas  causes  the  display  in  that  area  to 
switch  to  reverse  video  presentation.  Removing  the  finger  from 
the  display  triggers  activation  of  the  function  cited  in  that 
area.  The  Teledyne  display  output  was  the  conventional  ASCII 
character  set  with  special  modifications  to  accommodate  unique 
display  characters.  Also,  an  audible  alert  was  presented  to 
alert  the  aircrew  that  a  Data  Link  message  was  received  in  the 
cockpit.  Other  engineering  specifications  on  the  Teledyne  can 


TABLE  1.  PILOT  EXPERIENCE  SUMMARY 


General  Aviation 


Approx. 
Total  Hours 


Aircraft 


1  2500 

2  1100 

3  4950 


4 


1825 


E-2  T-34 ,  etc. 

C182  PA-28,  etc. 

C-421  Aero  Commander, 
Piper  Twin  E,  etc. 

BE 100  PA-31  DAC-6 ,  etc. 


Average  =  2600  Hours 


Airline 


Approx. 


Total  Hours 

Aircraft 

Primary 

Role 

Crew  1 

Captain 

Captain 

20000 

B-727 

& 

F/O 

First  officer 

5500 

B-727 

F/O 

Crew  2 

Captain 

Captain 

5000 

B-727 

First  officer 

25000 

B-727 

Captain 

Crew  3 

Captain 

6000 

B-727 

Captain 

First  officer 

n/a 

n/a 

n/a 

Crew  4 

Captain 

16000 

B-727 

B-737 

B-747 

Captain 

& 

F/0 

First  officer 

8000 

B-727 

DC- 10 

F/0 

Average  =  12200 
n/a  =  not  available 
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be  found  in  appendix  G.  Display  brightness  is  either  automatically 
controlled  with  an  ambient  light  sensor  or  manually  adjusted  by 
the  display  user. 

The  Teledyne  display  was  installed  in  the  NASA  B-727  in  the 
weather  radar  position.  The  Teledyne  display  was  installed  on 
the  GAT  instrument  panel,  centered  between  the  pilot  and  copilot 
position.  Photos  of  the  display  in  each  cockpit  are  provided  in 
appendix  G. 

3. 3. 2. 2  Data  Link  Services. 


The  package  of  services  for  the  airborne  evaluation  included  two 
ATC  and  four  WX  services.  The  ground/controller  evaluation 
included  two  additional  ATC  functions:  menu  text  and  free 
text.  The  menu  text  function  facilitates  the  prerecording  and 
transmission  of  repetitive  ground-to-air  messages.  The  free 
text  function  could  provide  one  line  (up  to  20  characters)  of 
brief  instructions  (ground-to-air  only) .  For  the  purposes  of  the 
airborne  evaluation  menu  text  provided  the  same 
information/instruction,  to  the  pilot  as  did  the  altitude 
assignment;  therefore,  data  on  these  two  services  were 
combined.  A  brief  description  of  each  of  the  tested  services 
is  presented  below.  Details  on  format  characteristics  and 
switching  logic  are  described  in  appendix  G. 

3. 3. 2. 2.1  Weather. 


Terminal  Forecast  (FT)  A  24  -  hour  prognosis  of  weather 
conditions  within  the  immediate  vicinity  of  a  selected  location; 
includes  sky  and  ceiling,  visibility,  weather,  obstructions  to 
vision,  cloud  heights,  and  whether  visual  or  instrument 
conditions  are  expected. 

Winds/Temperatures  Aloft  (FD)  -  A  report  of  projected  winds  and 
temperatures  for  a  range  of  altitudes;  includes  wind  speed,  wind 
direction,  and  temperature. 

Surface  Observation  (SA)  -  A  report  of  current  ground  weather  at 
a  selected  station;  includes  sky  conditions,  ceiling,  visibility, 
weather,  obstructions  to  vision,  wind  direction/speed,  altimeter 
setting,  and  additional  remarks. 

Pilot  Reports  (PIREPS)  (UA)  -  Pilot  reports  of  inflight 
conditions  which  may  include  information  on  sky  cover,  flight 
visibility,  flight  weather,  and  indications  of  icing  or 
turbulence,  etc. 
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3. 3. 2. 2. 2  Air  Traffic  Control. 


Altitude  Assignment  -  An  ATC  command  to  change  assigned  altitude. 
The  command  may  also  contain  time  or  crossing  references 
and,  if  required,  an  altimeter  setting. 

Frequency  Change  (or  transfer  of  communication)  -  An  ATC  command 
to  a  pilot  to  change  to  a  new  traffic  control  sector/facility 
frequency. 

3.3.3  Test  Scenarios. 


The  simulated  airspace  for  the  operational  evaluation  was  a 
section  of  the  FAA  UDS .  The  UDS  is  a  fictitious  airspace  created 
to  allow  simulations  of  the  NAS.  Four  experimental  scenarios  (two 
for  GAT  and  two  for  the  727)  were  created  for  this  evaluation  and 
are  discussed  briefly  below. 

In  test  scenario  1,  64  simulated  aircraft  flew  in  the  UDS.  The 
GAT  (designated  N8417P) ,  which  flew  at  low  altitudes  (12,000  feet 
and  below)  ,  was  one  of  the  64  aircraft.  The  remaining 
aircraft  (including  the  NASA  B-727,  designated  EAL55 ,  Eastern 
Airlines)  were  strictly  high  altitude  types  (between  12,000  and 
27,000  feet).  Six  different  airways  were  used  by  the  aircraft  in 
this  scenario. 

Test  scenario  2  contained  78  aircraft,  1  of  which  was  the  GAT 
and  1  the  NASA  B-727.  The  GAT  was,  again,  the  only  low  altitude 
aircraft.  Nine  different  airways  were  used  by  the  aircraft 
in  this  scenario. 

Each  data  run  consumed  approximately  30  minutes  of  operating 
time.  Time  between  each  data  run  was  approximately  15-20 
minutes.  Detailed  descriptions  of  the  GAT  and  B-727  scenarios 
are  provided  in  appendix  F. 

3.3.4  Test  Procedures. 

3. 3. 4.1  Training. 

The  subject  pilots  initially  received  a  complete  briefing  on  the 
simulator  they  were  to  fly.  Afterwards  they  were  briefed  on  the 
evaluation  and  trained  on  the  Data  Link  system.  Questions  were 
encouraged  to  insure  complete  understanding  of  all  aspects  of  the 
evaluation  and  systems  involved. 

3. 3.4.2  Testing. 

Prior  to  each  data  run,  the  subject  pilot/crew  was  given  flight 
plan  materials  (dispatch  papers)  as  required  and  informed  of  the 
run  details  (start  time,  scenario  number,  voice  or  Data  Link) . 
During  the  runs  the  subjects  were  cued  to  make  specific  weather 
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requests;  the  ATC  type  messages  were  acted  upon  as  required.  The 
scenario  descriptions  in  appendix  F  describe  the  events  in 
detail.  An  attempt  was  made  to  exercise  each  of  the  four  weather 
services  during  each  run.  The  number  and  type  of  ATC  messages 
were,  however,  variable.  A  breakdown  of  the  services  exercised 
in  each  run  is  provided  in  appendix  H. 

The  GAT  pilot  was  responsible  for  both  flying  the  simulator  and 
operating  the  Data  Link  system.  In  the  B-727  portion  of  the 
evaluation  a  two-person  crew  flew  the  simulator  (the  inflight 
observer  sat  in  the  Second  Officer/Flight  Engineer  seat) .  The 
Captain  was  responsible  for  flying  the  simulator  while  the 
First  Officer  (F/0)  operated  the  Data  Link  system.  The  Captains 
were  asked  to  monitor  the  operation  of  Data  Link  to  the  extent  safe 
operation  of  the  aircraft  would  allow. 

The  test  was  conducted  over  2  days.  Six  data  runs  were 
completed  each  day  at  each  location  for  a  total  of  24  runs.  Each 
pilot/crew  flew  three  runs;  two  with  Data  Link  and  one  using 
voice  (as  a  baseline  control  condition) .  The  counterbalanced 
sequence  of  runs  and  conditions  is  presented  below.  The  ground 
Data  Link  group  also  allowed  time  for  up  to  four  supplemental 
runs.  The  airborne  group  at  NASA  participated  in  one  of  these 
runs  because  of  a  technical  problem  in  a  data  run.  The  airborne 
group  at  the  FAA  Technical  Center  participated  in  two  of  these 
runs  (for  additional  data,  no  technical  problems  occurred). 

Randomized  Run  Sequence 

Run  Pilot/Crew  #  Voice/Data  Link  Scenario 


1 

1 

Voice 

1 

2 

1 

Data  Link 

1 

3 

1 

Data  Link 

1 

4 

2 

Data  Liiik 

2 

5 

2 

Data  Link 

2 

6 

2 

Voice 

2 

7 

3 

Data  Link 

1 

8 

3 

Data  Link 

1 

9 

3 

Voice 

1 

10 

4 

Voice 

2 

11 

4 

Data  Link 

2 

12 

4 

Data  Link 

2 

3.3.5  Data  Collection. 

3. 3. 5.1  Objective  Data. 

The  objective  measures  of  performance  during  the  evaluation  were 
various  time  intervals.  A  graphic  representation  of  the 
specified  time  intervals  and  an  example  script  of  voice-weather 
transactions  are  included  in  appendix  H.  Definitions  of  the 
critical  time  intervals  (in  both  Data  Link  and  voice)  follow. 


33 


The  inflight  observers  observed  the  operation  of  the  system  and 
obvious  errors/difficulties  on  the  part  of  the  subjects  were 
noted. 

The  aircrew  substudy  was  planned  to  provide  data  on  the  performance 
of  pilots  with  a  prototype  Data  Link  interface  system  in  the 
context  of  realistic  ATC  Data  Link  simulation.  The  time  intervals 
were  defined  in  such  a  way  as  to  provide  baseline  data  to  allow 
detailed  comparisons  with  evolving  displays  and  controls. 

3. 3.5. 1.1  Time  Interval  Definitions. 

3. 3. 5. 1.1.1  Weather. 

a.  MCT  -  Message  Creation  Time. 

Voice  -  The  interval  begins  with  the  pilot  activating 
the  mike  switch  (PTT-Push  to  Talk-ON) ,  during  which  a  request  for 
weather  is  made,  and  ends  when  the  pilot  releases  the  mike  switch 
(PTT-  OFF) . 

Data  Link  -  The  interval  begins  when  pilot  touches  one  of 
four  WX  products  on  the  MAIN  menu;  included  is  the  time  required 
in  selecting  the  appropriate  LOCID.  The  interval  ends  upon  touching 
"SEND"  in  the  LOCID  menu. 

b.  ST  -  System  Time. 

Voice  -  The  interval  begins  with  the  release  of  the  mike 
switch  in  MCT  and  ends  with  the  FSS/Flight  Watch  release  of  the 
mike  switch  after  announcing  that  WX  is  available,  i.e.,  "I  have 
your  request,  over." 

Data  Link  -  The  interval  begins  with  touching  "SEND"  in 
MCT  and  ends  when  "Touch  for  WX  Message"  is  displayed.  This 
interval  includes  simulated  random  Data  Link  transmission  delays. 

c.  RL  -  Response  Latency. 

Response  latency  denotes  that  this  interval  may  not  be 
readily  describable  as  a  reaction  time.  From  the  standpoint  of 
pilot  performance,  there  is  a  difference  between  receiving  the 
voice  message  and  receiving  the  one  on  a  controllable  visual 
display.  The  voice  message  did  not  persist,  but  the  visual  display 
remained  until  the  pilot  did  something  about  it.  The  response 
latency  measure  acknowledges  that  pilot  performance  may  be  quite 
different  with  a  stable  display.  Pilots  are  usually  performing  a 
variety  of  tasks  at  any  given  time.  When  some  new  stimulus 
appears,  they  may  not  be  free  to  attend  to  it  immediately. 
Instead,  they  tend  to  put  the  new  stimulus  in  a  queue  to  be 
attended  to  as  current  priorities  dictate.  It  is  useful  for  later 
uses  to  maintain  a  separate  identity  for  the  latency  data. 


Voice  -  The  interval  begins  with  the  FSS/Flight  Watch 
release  of  mike  switch  and  ends  when  the  pilot  presses  mike 
switch  to  announce,  "Yes,  Flight  Watch,  go  ahead." 

Data  Link  -  The  interval  begins  with  the  annunciation  of 
"Touch  for  WX  Message"  and  ends  when  the  pilot  touches  the  prompt 
"Touch  for  WX  Message." 

d.  MPT  -  Message  Processing  Time. 

Voice  -  The  interval  begins  when  FSS/Flight  Watch 
activates  the  mike  to  "read"  the  WX  message  to  the  pilot  and 
ends  when  FSS  releases  the  mike  after  reading  the  complete 
message.  Any  repeats  of  information  required  added  to  the  time 
as  appropriate. 

Data  Link  -  The  interval  begins  with  the  appearance  of 
requested  weather  on  the  screen  and  ends  when  the  pilot  touches 
MAIN  menu. 

3. 3. 5. 1.1. 2  Air  Traffic  Control. 

a.  RL  -  Response  Latency. 

Voice  -  No  counterpart  to  Data  Link. 

Data  Link  -  The  interval  begins  when  "Touch  for  ATC 
Message"  appears  on  the  display  and  ends  when  the  pilot  touches  the 
prompt  "Touch  for  ATC  Message." 

b.  WUT  -  WILCO/UNABLE  Response  Time. 

Voice  -  The  interval  begins  with  controller's  PTT-ON 
(during  which  the  ATC  instruction  is  delivered)  and  ends  with  the 
pilot's  release  of  the  mike  switch  after  acknowledging  the 
instruction.  If  the  pilot  requested  a  "SAY  AGAIN,"  this  time  was 
included . 

Data  Link  -  The  interval  begins  with  the  appearance  of 
the  ATC  Instruction  on  the  display  screen  and  ends  with  the 
touch  of  WILCO  or  UNABLE. 

3. 3.5.2  Subjective  Data. 

Subjective  data  were  collected  in  a  post-evaluation 
questionnaire.  The  questions  asked  of  the  subject  pilots  were 
grouped  around  each  particular  service  provided.  As  the  subject 
pilots/crews  were  completing  the  questionnaire,  a  human  factors 
engineer  was  available  to  answer  questions  and/or  discuss  issues. 
In  addition  to  the  questionnaires,  a  debriefing  session  was  held 
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with  the  GAT  subject  pilots,  controllers,  and  experimenters.  This 
session  was  held  the  day  after  data  collection  via  a  telephone 
conference  call.  The  B-727  crews  were  unable  to  participate  due 
to  schedule  constraints.  The  B-727  crews  were  given  the 
opportunity  to  debrief  with  this  same  group  immediately  after 
their  data  collection  runs  were  complete. 

3 .4  TEST  RESULTS. 


The  data  presented  are  from  24  data  runs;  12  runs  each  by 
general  aviation  and  airline  pilots.  The  data  presented  are 
specific  to  the  flight  deck.  Controller/ground  oriented  data  are 
presented  in  section  2. 

It  should  be  noted  that  when  more  than  one  request  for  a  given 
weather  service  was  accomplished  in  a  single  run,  the  data  were 
not  considered  in  the  analysis  to  ensure  balance.  The  B-727  run  2 
data  were  gathered  from  a  supplemental  type  run  because  of  a 
technical  problem  in  the  actual  run  2. 

3.4.1  Objective  Data. 

Various  time  intervals  were  measured  and  subsequently  reduced 
into  descriptive  statistics  to  show  means  and  variability. 
Breakdowns  according  to  pilot  type  (general  aviation  or  airline) , 
scenario  type,  and  COMM  type  (Data  Link  or  voice)  will  be 
provided.  Note  that  in  the  ATC  voice  condition,  no  response 
latency  is  definable.  No  inferential  tests  were  conducted  on  the 
time  data  because  of  the  small  sample  size  (number  of  pilots  and 
runs)  . 

Figure  5  presents  the  pilot  time  response  data  for  overall  ATC 
WILCO  or  UNABLE  time  differences  between  Data  Link  and  voice 
by  GAT  and  B-727.  The  overall  weather/pilot  response  time  data 
by  Data  Link  and  voice  (by  GAT  and  B-727)  are  presented  in  figure 
6.  The  overall  time  data  (average  and  standard  deviation)  for 
WX/ ATC  for  GAT  and  B-727  are  presented  in  table  2.  Time  data  for 
each  individual  ATC  and  WX  service  are  provided  in  appendix  H. 

3.4.2  Subjective  Data. 

The  questionnaire  was  administered  to  four  general  aviation 
pilots  and  four  airline  crews.  A  total  of  12  questionnaires  was 
obtained.  The  questionnaire  was  divided  into  three  parts:  ATC 
Services  (questions  1-11);  WX  Services  (questions  12-19);  and, 
Overall  (questions  20-30) .  The  numerical  average  (Avg)  and 
standard  deviation  (Std)  was  calculated  for  each  rating  type 
answer.  The  Avg  and  Std  was  calculated  for  the  GAT  group,  the 
B-727  group,  and  the  combination  of  groups.  The  standard  rating 
scale  used  in  many  of  the  questions  is  presented  below. 
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DATALINK  VS.  VOICE 
Overall  -  Weather 


25  - 


MCI  ST  RL 

Time'  Interval 


N  DATALINK  -  GAT  ^  VOICE  -  GAT 

CZ 3  DATALINK  -  8-727  H  VOICE  -  B-727 


FIGURE  6.  WEATHER-PILOT  TIME  DATA 
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GAT 

B-727 


GAT 

B-727 


GAT 

B-727 


TABLE  2.  GAT/B-727  OVERALL  TIME  DATA 

Time  in  Seconds 
Data  Link  -  Weather 

MCT  ST  RL  MPT 


Avg 

Std 

Avg 

Std 

Avg 

Std 

Avg 

Std 

7 . 6 

9 . 1 

14 . 5 

1.7 

3 . 6 

1.4 

10.3 

3 . 6 

7.1 

4 . 1 

15.9 

4.8 

3 . 7 

0.7 

15.0 

6.6 

Voice  -  Weather 


MCT  ST  RL  MPT 


Avg 

Std 

Avg 

Std 

Avg 

Std 

Avg 

Std 

7 . 5 

2.6 

20.4 

1.4 

2.9 

1.1 

17 . 8 

3.7 

9.9 

3.9 

8.4 

3.6 

2 . 0 

0.6 

17 . 1 

8 . 6 

Data  Link  -  ATC 


RL  WUT 

Avg  Std  Avg  Std 

3.7  0.6  5.8  1.8 

4.4  3.8  7.9  3.3 

Voice  -  ATC 


WUT 

Avg  Std 
9.9  3.1 

9.7  5.0 

n/a  =  Not  applicable 


RL 

Avg  Std 

GAT  n/a 

B-727 
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Standard  Scale 


7  -  VERY  GOOD,  no  changes  are  necessary 
6  -  GOOD,  a  few  minor  changes  are  necessary 
5  -  FAIRLY  GOOD,  a  number  of  minor  changes  are  necessary 
4  -  FAIR,  both  minor  and  major  changes  are  necessary 
3  -  FAIRLY  POOR,  a  few  major  changes  are  necessary 
2  -  POOR,  a  number  of  major  changes  are  necessary 
1  -  VERY  POOR,  a  complete  redesign  is  necessary 

The  following  three  sections  summarize  the  individual  sections  of 
the  questionnaire.  Detailed  analysis  of  the  subjective  results 
can  be  found  in  appendix  H. 

3.4.2. 1  Specific  Results  Regarding  the  ATC  Services. 

For  each  ATC  Service,  altitude  assignment,  and  frequency  change 
subject  pilot  ratings  were  obtained  on  six  factors:  Appearance, 
Clutter,  Amount  of  Information,  Information  Clarity,  Ambiguity, 
and  Time  to  Interpret.  On  all  factors  the  average  ratings  were 
GOOD  (6)  or  above  with  B-727  pilots  rating  the  factors  slightly 
lower  than  GAT  pilots. 

In  general,  all  pilots  were  satisfied  with  the  auditory 
annunciation  of  incoming  ATC  Data  Link  messages.  Two  B-727 
pilots  expressed  concerns  with  high  workload  situations  and 
the  startling  effect  the  tone  had.  Other  comments  were  made 
concerning  the  tone  in  a  real  cockpit,  missing  the  tones,  and 
number  of  tones  in  the  present  cockpit. 

The  visual  annunciation  of  ATC  Data  Link  messages  was  considered 
acceptable.  One  pilot  suggested  that  flashing  the  entire  screen 
and  providing  a  Master  Caution,  e.g. ,  light  would  help  attract  the 
pilot's  attention.  The  other  pilot  suggested  a  change  in 
terminology  of  the  ATC  message  prompt  "TOUCH  FOR  ATC  MESSAGE"; 
he  added  that  "TOUCH"  was  not  necessary.  Negative  responses 
were  obtained  for  the  automatic  appearance  of  ATC  messages  in 
which  there  is  no  ATC  message  prompt.  Pilots  felt  that  automatic 
delivery  would  be  too  easy  to  neglect  or  miss  altogether  and 
possibly  override  an  existing  message,  such  as  a  WX  report,  which 
they  hadn't  finished  reading. 

The  "SENT"  indication  after  a  WILCO  or  UNABLE  was  considered 
desirable  by  most  of  the  pilots.  One  pilot  stated  that  if  the 
logic  was  such  that  WILCO/UNABLE  did  not  disappear  until  the 
system  accepted  the  "SENT"  command,  the  display  of  "SENT"  would 
be  unnecessary.  In  general,  a  confirmation  that  the  system 
accepted  the  input  was  considered  necessary. 

For  both  altitude  assignments  and  frequency  changes,  the  time 
diverted  from  flying  to  operate  Data  Link  was  considered 
acceptable  (or  minimal) .  One  pilot  who  was  undecided  added  that 
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verbal  commands  may  be  quicker  in  certain  situations,  e.g., 
instrument  conditions  or  approach  to  a  buuy  airport.  One  pilot 
stated  that  voice  communications  also  required  diversion  from 
flying  (to  the  same  extent  as  Data  Link)  and  that,  overall,  Data 
Link  is  superior  to  writing  clearances,  etc. 

The  ATC  Data  Link  overall  design  was  rated  FAIRLY  GOOD  (5)  to 
GOOD  (6).  The  comments  were  very  positive,  e.g.,  "...better  than 
I  thought  it  would  be  at  this  stage  of  development,"  "...like 
the  Data  Link  just  the  way  it  is."  Suggested  changes  to  the 
present  design  dealt  with  terminology,  number  of  services, 
information  flow  rates,  etc. 

3. 4. 2. 2  Specific  Results  Regarding  the  WX  Services. 

As  with  the  ATC  Services,  subject  pilots  rated  each  of  the  four 
WX  Services:  Surface  Observation,  Terminal  Forecast,  Pilot  Report, 
and  Wind/Temp  Aloft,  using  the  standard  rating  scale.  In  all, 
eight  factors  for  each  service  were  rated.  In  addition  to  those 
factors  listed  for  ATC  services  in  section  3.4.2. 1 
(excluding  Time  to  Interpret) ,  Information  Retrieval  Time,  Time 
or  Valid  Times  Concept,  and  "Return  to  Main"  concept  were 
included.  In  general,  all  average  ratings  were  FAIRLY  GOOD  or 
above  (>5).  The  standard  deviations  were  fairly  high,  indicating 
a  wider  range  of  ratings.  Several  comments  were  made  by  the 
pilots  concerning  general  format,  content,  amount  of  information, 
and  logic. 

During  Data  I, ink  runs  requests  for  WX  services  were  made  through 
the  Location  Identifier  menu  (LOCID) .  Preselected  stations  (made 
by  the  experimenters)  appeared  on  the  LOCID  menu  after  pilots 
requested  the  WX  Service  desired  from  the  MAIN  menu.  Using  the 
standard  rating  scale,  the  subject  pilots  rated  the  LOCID  menu  on 
six  factors:  Menu  of  LOCID' s  Concept;  Time  Required  to 
Input/Enter,  Possibility  for  Error,  Number  of  "Keystrokes," 
LOCID  menu  of  Location  Codes,  and  LOCID  Remaining  on  Format. 
The  factors  were,  in  general,  rated  higher  by  GAT  pilots  than 
B-727  pilots.  The  LOCID  menu  concept  and  time  to  input 
factors  were  rated  lower  by  B-727  pilots.  The  menu  of  LOCID 
codes  was  rated  FAIR  (4)  by  both  groups.  The  comments  centered 
on  actual  implementation,  number  of  LOCID' s,  ambiguity  of  LOCID 
codes,  etc. 

The  "SENT"  message  after  touching  SEND  was  considered  a  very 
desirable  option.  The  confirmation  it  provides  that  the  system 
is  working  is  critical. 

The  logic  in  which  weather  information  remains  on-screen  until 
pilot  input  was  considered  very  desirable.  The  ability  to  read 
the  message  more  than  once  and/or  copy  it  onto  paper  is 
necessary . 
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For  all  WX  Services,  the  time  diverted  from  the  flying  task  was 
considered  acceptable  by  most.  Although  one  pilot  felt  that 
improved  formats  would  provide  a  quicker  response  and  greatly 
improve  the  service. 

The  overall  Weather  Data  Link  design  was  rated  FAIRLY  GOOD. 
Problem  areas  listed  were  centered  on  the  information 
presentation,  e.g.,  letter  size,  amount,  etc. 

3. 4. 2. 3  General  Overall  Issues. 


For  both  ATC  and  WX  Services,  subject  pilots  were  asked  to  rate 
the  workload  for  the  services  as  they  appeared  during  the  flight 
using  the  scale  as  presented  below. 

5  Completely  saturated 
4  High  workload 

3  Acceptable  at  all  times,  not  too  high  OR  low 
2  Low  workload 

1  Very  low,  bored,  need  activity 

The  ATC  workload  ratings  given  by  GAT  pilots  (acceptable)  were 
almost  a  full  point  higher  than  those  given  by  B-727  pilots  (low 
workload) .  This  may  be  due  to  the  fact  that  the  GAT  pilot  was 
alone,  whereas,  the  B-727  portion  utilized  a  two-person  crew. 
Ironically,  however,  the  GAT  pilots  rated  the  workload  lower  for 
the  WX  Services. 

After  entering  a  weather  request,  the  system  remained  in  the 
LOCID  menu.  Upon  delivery  of  the  requested  service  to  the  Data 
Link  system,  a  message  appeared  at  the  bottom  of  the  display 
indicating  its  availability.  Upon  touching  the  area,  the 
weather  information  was  presented.  The  logic  flow  associated 
with  the  LOCID  menu/weather  retrieval  was  acceptable  by  all.  One 
pilot  did  not  want  to  have  to  touch  to  retrieve  the  weather  when 
available . 

3.5  DISCUSSION. 

This  evaluation  represents  a  significant  step  toward  the 
implementation  of  a  usable  airborne  Data  Link  system.  Qualified 
pilots  were  used  to  examine  an  ATC  and  WX  system  in  real-time 
situations.  Thus,  consideration  of  the  impact  upon  real 
flightcrews  was  undertaken  at  a  stage  of  development  that  will 
permit  meaningful  response  to  the  needs,  desires,  and  limitations 
of  operational  personnel.  The  results  showed  that  Data  Link  can 
be  a  time  saver.  However,  care  must  be  taken,  as  pointed  out  by 
the  pilots,  to  avoid  introducing  new  problems  (e.g.,  visual 
annunciation,  party  line  loss,  etc.). 

Overall,  the  data,  both  objective  and  subjective,  indicate  that 
Data  Link  may  enhance  the  communication  link  between  pilots  and 
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the  ground  (ATC  and  WX) .  Although  the  number  of  ATC  Services 
was  limited,  the  objective  data  may  be  generalizable  across 
other  routine  ATC  Services.  The  primary  dependent  variable  (DV) 
with  ATC  Services,  WILCO/UNABLE  Time,  would  probably  be  related 
to  the  complexity  of  the  instruction.  Therefore,  for  routine 
instructions  the  Data  Link  -  voice  (WILCO/UNABLE)  time 
differences  should  be  comparable  to  the  observed  data.  Only 
further  testing  will  determine  the  validity  of  this  assumption. 

Examination  of  the  ATC  time  data  (WILCO/UNABLE  time)  will  show 
that,  overall  and  for  each  service  individually  (appendix  H) ,  the 
time  to  interpret  and  respond  to  messages  was  shorter  with  Data 
Link.  This  tendency  occurred  with  both  GAT  and  B-727  pilots. 
The  Data  Link  -  voice  time  differences  were  consistently  larger 
with  the  GAT  pilots.  For  the  GAT,  the  percentage  differences 
in  WILCO/UNABLE  time  between  Data  Link  and  voice  in  altitude 
assignment,  frequency  change,  and  free  text  cases  were, 
respectively,  40,  43,  36,  and  38.  Conversely,  the  B-727  time 
data  indicate  percentage  differences  of  only  18,  33,  7  and  5. 

This  difference  in  favor  of  the  GAT  pilots  should  be  interpreted 
cautiously.  It  is  frequently  a  mistake  for  external  observers  to 
attach  importance  to  a  particular  time  measure  of  aircrew 
performance.  One  must  first  determine  whether  the  aircrew  members 
themselves  felt  that  performance  time  to  be  important.  Pilots  have 
many  duties  to  perform.  Any  particular  function  is  attended  to  in 
the  context  of  all  the  other  things  to  be  done.  One  particular 
function  may  seem  important  to  the  external  observer,  but  the  pilot 
tends  to  select  out  those  of  highest  priority  at  the  moment.  How 
quickly  they  do  perform  a  particular  function  may  not  be  related 
to  how  quickly  they  can  perform  it. 

The  experimenters  did  make  some  observations  about  pilot  behavior 
during  the  test  runs.  Contrary  to  instructions,  a  few  of  the  B- 
727  pilots  would  act  on  an  ATC  command  before  pressing  WILCO  (or 
UNABLE),  e.g.,  they  would  change  the  altitude  hold  setting  of  the 
autopilot  or  change  the  voice  radio  frequency,  then  WILCO  the 
message.  The  larger  response  times  may  be  related  to  the  "two- 
person"  operation.  In  a  single-pilot  operation,  the  pilot  is  the 
only  human  component.  Conversely,  in  the  airline  case,  the  First 
Officer  must  decipher  the  message,  then  vocalize  it  to  the  Captain, 
who  then  makes  a  decision,  tells  the  First  Officer  to  either  WILCO 
or  UNABLE.  The  airline  environment  results  in  an  additional 
component  in  the  information  loop,  i.e.,  the  First  Officer  is  not 
present  in  the  single-pilot  environment.  The  crew  coordination 
process  (in  this  test)  appears  to  have  added  to  the  response  time. 

The  weather  data  are  not  as  conclusive  as  the  ATC  data. 
Examination  of  the  overall  data  will  show  that  message  creation 
times  (MCT)  were  essentially  equal  (Data  Link  vs.  voice,  GAT  vs. 
B-727) .  The  B-727  voice  condition  took  slightly  longer.  The 
DV  of  overall  message  processing  time  (MPT)  favors  Data  Link  (both 
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GAT  and  B-727) .  The  Data  Link-voice  MPT  time  differences  for  GAT 
and  B-727  are,  42  and  12  percent,  respectively.  The  response 
latency  DV  favors  voice,  both  GAT  and  B-727. 

System  time  for  Data  Link  was  essentially  equal  between  GAT  and 
B-727.  System  time  for  voice  between  GAT  and  B-727  was 
considerably  different.  This  difference  is  related  to  the  fact 
the  voice  weather  came  from  two  different  individuals.  The 
individuals  were  to  receive  the  request,  wait  for  approximately 
10-15  seconds  (as  if  getting  the  information  from  an  on-line 
computer) ,  then  read  the  information  to  the  pilot.  Individual 
differences  in  wait  time  probably  account  for  this  time 
difference. 

The  DV's,  MCT,  and  MPT  are  considered  most  critical  in  this 
discussion  because  they  are  pilot  specific.  The  Response  Latency 
(RL)  variable  is  pilot  specific;  but,  because  of  the  small  times 
involved,  is  not  as  critical  (in  the  test  engineers' 
opinions) .  Given  this  assumption,  further  examination  of  the 
GAT  data  will  show  MCT  essentially  the  same  in  Data  Link  and 
voice  overall  and  in  each  of  the  services  individually.  GAT  MPT 
differences  between  Data  Link  and  voice  are  relatively  large. 
Percentage  differences  for  MPT  overall,  terminal  forecasts, 
surface  observations,  wind/temperature,  and  PIREP's  were  42, 
51,  37,  35,  41,  respectively. 

Again,  accepting  the  above  criticality  assumption,  examination  of 
the  B-727  data  will  show  that  MCT  overall  with  voice  requires  28 
percent  more  time  than  Data  Link.  For  terminal  forecasts, 
surface  observations  and  wind/temp  aloft,  respectively,  the  voice 
condition  required  49,  19,  and  38  percent  more  time  than  Data 
Link.  B-727  MPT  percentage  difference  between  Data  Link  and 
voice  for  overall  terminal  forecasts,  surface  observations,  and 
wind/temperatures  were  12,  20,  7,  and  32,  respectively.  There 
were  no  voice  PIREP's  accomplished  by  the  B-727  pilots. 

In  examining  the  time  data,  consideration  must  be  given  to  the 
fact  that  eight  Data  Link  runs  and  four  voice  runs  were 
accomplished.  This  inequality  in  number  of  runs  may  contribute 
to  the  differences.  Also,  one  must  be  careful  of  GAT  B-727 
comparisons  because  (a)  the  GAT  and  B-727  flew  in  two  different 
altitude/traffic  environments,  and  (b)  the  B-727  time  data  were 
obtained  from  a  non-flying  operator  of  Data  Link;  whereas,  the  GAT 
pilot  flew  and  operated  Data  Link. 

A  point  for  the  reader  to  consider  in  evaluating  the  weather  time 
data,  specifically,  message  creation  time,  is  presented.  In  a 
voice  environment,  where  an  aircraft  only  has  one  radio,  a 
frequency  change  to  a  Flight  Service  Station  (Flight  Watch)  must 
occur.  Conversely,  in  Data  Link,  the  pilot  simply  inputs  the 
request.  The  point  -  -  message  creation  time  in  voice  could 
justifiably  be  defined  as  the  time  from  when  the  pilot  starts  the 
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request  to  ATC  to  be  off  the  air  (obtaining  weather)  through 
changing  the  frequency,  and  finally,  calling  for  the  weather.  If 
this  definition  of  message  creation  time  were  accepted,  instead 
of  the  definition  applied  to  the  data  (defined  earlier  in  this 
report) ,  the  difference  in  time  would  be  emphasized.  The  authors 
chose  the  conservative  definition  in  analyzing  the  data  to  avoid 
possible  differences  of  opinion.  Nonetheless,  the  point  is 
presented  for  consideration. 

Other  points  to  consider  are:  a  limited  number  of  services  were 
presented,  thus,  limiting  judgments  on  workload  (as  in  a  fully 
implemented  system) ,  the  frequency  of  voice  callbacks  to  clarify 
information  which  resulted  in  longer  WILCO/UNABLE  time,  and 
equipment  constraints  limited  the  system  capability.  Despite 
these  constraints,  there  were  substantial  time  savings  for  Data 
Link  over  voice.  This,  in  itself,  would  warrant  furtherance 
of  this  concept.  In  support  of  the  time  data,  pilot  perceptions 
of  the  value  of  Data  Link  were  very  positive.  Together,  the 
time  data  and  pilot  perceptions  emphasize  the  validity  of  the 
Data  Link  concept. 

4.  JOINT  CONTROLLER  AIRCREW  ISSUES. 

The  two  substudies  presented  in  this  report  used  a  common 
simulation  environment  to  address  human  interface  and  procedural 
design  issues  relevant  to  the  roles  of  aircrew  and  air  traffic 
controllers  in  producing  an  effective  implementation  of  Data  Link. 
Many  of  these  issues  are  concerned  with  problems  that  are  unique 
to  either  the  airborne  or  ground  elements  of  the  system,  and  are 
essentially  transparent  to  controllers  or  their  pilot  counterparts 
working  in  separated  environments.  For  this  reason,  the  two 
substudies  were  aesigned  and  conducted  in  a  relatively  independent 
manner. 

However,  because  a  significant  determinant  of  the  success  of  Data 
Link  will  be  the  extent  to  which  effective,  two-way  communications 
can  be  accomplished,  the  operational  evaluation  also  include 
measures  and  limited  debriefing  discussions  which  emphasized  the 
interaction  between  controllers  and  pilots  that  will  occur  over 
this  system.  This  section  of  the  report  draws  upon  the  individual 
results  of  both  substudies,  prior  findings  from  the  controller  mini 
studies,  and  preliminary  contacts  with  pilot  organizations  to 
examine  several  joint  design  issues. 

4.1  DATA  LINK  AS  A  PRIMARY  SERVICE  DELIVERY  MODE. 

Two  concepts  were  originally  proposed  for  the  application  of  Data 
Link  to  the  delivery  of  ATC  services.  One  of  these  designated  Data 
Link  as  a  primary  delivery  mode  which  would  act  as  a  substitute  for 
voice  messages.  The  alternative  approach  viewed  Data  Link  as  a 
secondary  mode  in  which  a  digital  uplink  would  be  redundant  with 
a  voice  clearance.  When  both  of  these  concepts  were  presented  to 
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controllers  during  the  preliminary  mini  studies,  the  subjects  were 
unanimous  in  their  preference  for  the  use  of  Data  Link  as  a  primary 
communications  method  which  would  be  fully  interchangeable  with 
voice.  The  general  consensus  was  that,  in  order  to  reduce 
frequency  congestion  and  keep  controller  workload  within  acceptable 
bounds.  Data  Link  would  have  to  be  sufficiently  reliable  to 
eliminate  redundant  voice  messages.  The  sole  precaution  suggested 
by  the  controllers  was  that  voice  radio  linkage  must  be  maintained 
as  a  back-up  to  Data  Link  and  as  a  means  to  deliver  messages  that 
are  inappropriate  for  digital  transmission.  Subsequent  studies, 
including  the  operational  evaluation,  were  conducted  without 
redundant  voice  transmission  and  no  controller  inputs  have  been 
received  which  contradict  this  philosophy. 

In  general,  it  appears  that  pilots  agree  with  the  use  of  Data  Link 
as  a  primary  service  delivery  method.  Comments  received  from 
aircrew  who  participated  in  the  operational  evaluation  indicated 
that  Data  Link  should  be  able  to  serve  all  communication  functions 
currently  performed  using  voice  radio,  and  that  Data  Link  can  be 
expected  to  improve  the  accuracy  and  reduce  the  workload  of  ground- 
air  communications. 

4.2  REQUIREMENT  FOR  A  DOWNLINKED  WILCO  TO  ATC  MESSAGES. 

As  dictated  by  the  results  of  the  preliminary  controller  mini 
studies  and  by  current  ATC  doctrine,  the  Data  Link  procedures  used 
in  the  operational  evaluation  study  required  pilots  to  downlink  a 
WILCO  response  to  confirm  receipt  of  a  message  and  an  intent  to 
carry-out  the  ATC  instruction.  The  controller  philosophy  behind 
this  requirement  is  that  a  message  is  not  considered  delivered  to 
the  flight  deck  unless  it  is  followed  by  a  confirmatory  response. 
This  concept  was  embodied  in  the  software  logic  developed  for  Data 
Link  by  making  receipt  of  the  WILCO  response  the  key  event  which 
updates  information  about  an  aircraft  in  the  NAS  data  base. 

The  pilots  who  participated  in  the  operational  evaluation  appeared 
to  agree  that  the  downlinked  response  is  an  appropriate  means  for 
indicating  their  intent  to  comply  with  an  ATC  instruction. 
Overall,  the  response  concept  received  an  average  rating  of  6.27 
(1-7  scale) ,  indicating  a  high  level  of  acceptance. 

4.3  THE  UNABLE  DOWNLINK. 


Early  controller  inputs  to  the  tested  designs  for  Data  Link 
suggested  that  the  receipt  of  a  downlinked  from  an  aircraft  as  an 
indication  that  the  pilot  could  not  comply  with  an  instruction  was 
an  insufficient  response.  Some  of  the  controllers  felt  that  the 
capability  to  send  this  response  should  be  eliminated  from  a  Data 
Link  cockpit  hardware  design.  The  rationale  underlying  this 
recommendation  was  that  the  message  was  uninformative  and  time 
consuming,  and  should  be  replaced  by  a  procedure  requiring  an 
immediate  radio  contact  from  the  pilot. 
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Informal  discussions  with  pilots  prior  to  the  operational 
evaluation  suggested  that  there  could  be  some  resistance  to 
eliminating  the  UNABLE  key,  presumably  because  this  would  restrict 
downlinks  to  confirmation  responses.  During  the  operational 
evaluation  pilots  were  instructed  to  use  both  the  UNABLE  downlink 
and  an  immediate  radio  call  to  the  controller  to  resolve  any 
problem.  Although  not  directly  addressed  by  any  evaluation 
instruments,  this  procedure  for  handling  exceptions  did  not  prompt 
negative  comments  from  controllers  or  pilots. 

4.4  DATA  LINK  DELAYS. 

As  noted  in  the  results  of  the  controller  substudy,  Data  Link 
delays  associated  with  radar  antenna  rotation  did  not  affect 
controller  workload  or  result  in  negative  comments;  even  when  these 
delays  were  extended  to  twice  those  predicted  for  an  operational 
Mode  S  system.  While  the  tested  time  lags  could  be  relatively 
long,  controllers  appeared  to  adjust  to  the  system  quite  rapidly, 
either  by  issuing  clearances  in  a  more  anticipatory  fashion  or  by 
switching  to  voice  in  time-critical  situations. 

Pilots  in  the  operational  evaluation  were  also  largely  unconcerned 
with  Data  Link  turn  around  times  for  ATC  clearances.  This  response 
was  predictable  since  the  aircrew  reacts  to  an  ATC  message  only 
after  it  arrives  on  the  airborne  display,  and  is  unaware  of  the 
total  delay  which  transpires  from  the  issuance  of  a  clearance  to 
the  receipt  of  a  confirmatory  response  by  the  controller. 
Nevertheless,  the  fact  that  pilots  did  not  notice  an  effect  of  Data 
Link  delays  for  ATC  messages  suggests  that  these  lags  had  no 
negative  impact  on  the  perceived  timeliness  of  controller 
instructions. 

The  pilots  did  comment  on  the  delay  which  elapsed  between  a  request 
for  weather  information  and  its  receipt  in  the  cockpit.  For  these 
services,  where  the  aircrew  was  cognizant  of  the  full  cycle  of 
downlinks  and  uplinks,  delays  were  considered  potentially 
problematic. 

4.5  INITIAL  CALL. 

In  the  current  ATC  system  pilots  are  required  to  call  the 
controller  on  the  newly  assigned  frequency  after  they  receive  a 
transfer  of  communication  message.  Controllers  who  participated 
in  the  design  and  evaluation  of  the  Data  Link  transfer  of 
communication  service  have  suggested  that  this  initial  call  could 
be  eliminated  since  the  pilot  will  have  a  verified,  recallable 
display  of  the  new  radio  frequency  available  in  the  cockpit  at  all 
times.  From  the  point  of  view  of  the  controller,  removing  the 
requirement  for  initial  call  would  reduce  workload  while 
maintaining  positive  contact  with  each  aircraft. 
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Informal  discussions  with  pilots  and  unsolicited  comments  from  the 
operational  evaluation  indicate  that  aircrews  may  be  somewhat 
reluctant  to  relinquish  this  radio  contact.  The  general  opinion 
was  that  the  initial  call  serves  to  verify  the  radio  link  and 
assure  the  pilot  that  the  aircraft  is  under  positive  ATC 
surveillance  in  the  new  sector.  Controller  responses  to  this 
comment  suggest  that  the  initial  call  only  verifies  the  existence 
of  the  radio  link  during  the  call  itself,  and  that,  in  any  case, 
the  pilot  has  the  back-up  assurance  of  the  Data  Link  connection. 

Although  no  consensus  was  achieved  on  this  issue,  it  appears  that 
the  disagreement  may  be  resolved  as  pilot  confidence  in  the 
integrity  and  reliability  of  Data  Link  is  gained  during  the  system 
implementation  process.  In  the  transitional  phase,  both 
controllers  and  pilots  appear  to  feel  that  the  initial  call  may  be 
retained.  Additionally,  elimination  of  the  initial  call  will 
necessitate  reconsideration  of  operational  procedures  since,  under 
some  conditions,  this  transaction  is  currently  required  as  a  check 
on  an  aircraft's  assigned  altitude  and  on  the  Mode  C  readout. 

4.6  LOSS  OF  THE  "PARTY  LINE". 

A  general  concern  expressed  by  many  pilots  in  their  informal 
considerations  of  Data  Link  and  in  comments  recorded  during  the 
operational  evaluation  was  that  this  discretely-addressable 
communications  system  may  sacrifice  an  important  source  of  flight 
information.  Although  the  ability  to  receive  only  those  messages 
directed  to  one's  own  aircraft  will  help  to  eliminate  confusion  and 
the  need  to  monitor  all  broadcasted  voice  transmissions,  pilots 
feel  that  this  feature  of  Data  Link  could  result  in  the  loss  of 
a  source  of  general  knowledge  about  other  air  traffic  in  the 
vicinity,  and  advance  information  about  local  weather  aloft  and  ATC 
clearances  that  can  be  expected  in  the  near  future.  For  many 
pilots  this  "party  line"  currently  provided  by  voice  transmissions 
appears  to  be  an  important  basis  for  maintaining  situation 
awareness. 

Approached  with  the  problem,  controllers  have  argued  that  use  of 
the  incidental  information  from  the  party  line  often  can  lead 
pilots  to  take  inappropriate  anticipatory  action  and  that  a 
"silent"  Data  Link  environment  is  unlikely  to  be  experienced  by 
pilots  in  the  foreseeable  future.  In  addition,  some  controllers 
have  projected  that  as  Data  Link  begins  to  relieve  voice  frequency 
congestion,  controllers  will  be  able  to  use  voice  to  provide  pilots 
with  more  direct  data  to  support  their  knowledge  of  the  current  and 
expected  flight  environment. 

Pilots  also  made  comments  regarding  possible  ameliorating  factors. 
In  particular,  they  have  noted  that  the  importance  of  the  party 
line  may  diminish  as  more  situation  data  are  made  available  to  the 
cockpit  using  systems  such  as  the  Traffic  Advisory  and  Collision 


48 


Avoidance  System  (TCAS) ,  or  from  automatically  uplinked  information 
about  surrounding  aircraft  positions  from  the  NAS  data  base. 

4.7  CONDITIONS  FOR  REVERSION  TO  VOICE. 

Although  pilots  and  controllers  agreed  that  the  use  of  Data  Link 
for  ATC  clearances  was  appropriate,  both  groups  indicated  that 
there  are  conditions  under  which  voice  radio  should  be  used 
exclusively  for  transmission  of  control  instructions.  The  majority 
of  these  situations  involved  communications  which  were  time- 
critical,  where  potential  Data  Link  delays  may  be  intolerable.  In 
addition  to  emergency  calls,  recommended  conditions  for  mandatory 
voice  contact  included  busy  terminal  area  final  approach  and 
arrival,  and  other  cases  where  the  pilot's  visual  attention  is 
focused  outside  of  the  aircraft. 

While  pilots  and  controllers  tended  to  concur  on  this  issue,  it 
should  also  be  noted  that  the  speed  of  Data  Link  communication  and 
the  ease  with  which  aircrew  can  interpret  and  utilize  uplinked 
information  may  vary  with  the  specific  Data  Link  transmission 
medium  (e.g..  Mode  S  vs.  satellite)  and  the  design  of  cockpit 
interfaces  (e.g.,  text  vs.  graphics  or  synthesized  voice).  Thus, 
it  is  possible  that  the  boundary  conditions  for  the  use  of  Data 
Link  may  change  as  these  technologies  evolve. 

4.8  DISCUSSION. 

Examination  of  the  data  presented  in  this  section  of  the  report 
suggests  that  although  controllers  and  aircrew  appear  to  agree  on 
many  topics,  several  issues  governing  ground-air  interaction  over 
Data  Link  remain  unresolved.  Most  of  these  issues  appear  to  be 
concerned  with  problems  that  can  be  addressed  through  the 
development  of  jointly  determined  controller  and  pilot  operational 
procedures  rather  than  major  hardware  or  software  modifications  to 
Data  Link. 

5.  CONCLUSIONS. 

The  results  of  the  study  presented  in  this  report  warrant  a  number 
of  conclusions  about  the  ground  and  airborne  components  of  the  Data 
Link  system.  Conclusions  derived  from  the  results  of  the 
controller  and  aircrew  substudies  are  listed  below  in  separate 
sections.  A  final  conclusion  addresses  those  issues  which  impact 
the  interaction  between  controllers  and  pilots. 

5.1  CONTROLLER  SUBSTUDY. 


a.  General  perceptions  of  Data  Link  recorded  during  the 
operational  evaluation  were  in  agreement  with  the  findings  of 
earlier  mini  design  studies.  The  controller  participants  indicated 
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that  implementing  Data  Link  in  operational  en  route  field 
environments  will  have  positive  effects  on  the  overall  safety  and 
efficiency  of  ATC  and  on  system  capacity. 

b.  The  results  of  the  study  provided  the  first  empirical 
evidence  of  the  extent  to  which  Data  Link  can  be  expected  to 
alleviate  the  problem  of  voice  radio  frequency  congestion  in  the 
ATC  system.  Using  only  the  transfer  of  communication,  altitude 
assignment,  and  text  functions  included  in  the  initial  Data  Link 
service  package,  individual  voice  transmissions  and  channel 
occupation  time  were  significantly  reduced  in  comparison  to  tests 
in  which  current,  voice-only  procedures  were  used. 

c.  Comparisons  of  current  and  Data  Link  procedures  indicated 
that  in  addition  to  a  reduction  in  voice  radio  usage,  Data  Link  may 
increase  the  overall  efficiency  of  controller-pilot  communication 
by  reducing  requirements  for  resending  garbled  and  misinterpreted 
messages . 

d.  Data  Link  usage  in  low  and  high  traffic  portions  of  the 
test  scenarios  indicates  the  tested  services  are  likely  to  be 
equally  effective  across  a  broad  range  of  operational  situations. 

e.  The  observed  benefits  of  Data  Link  to  Air  Traffic  Control 
(ATC)  productivity  were  not  gained  at  any  measurable  loss  in  ATC 
safety  or  increase  in  the  perceived  workload  of  the  test 
controllers.  Because  the  subject  controllers  received  only  basic 
training  with  the  Data  Link  services  and  were  unfamiliar  with  the 
ATC  test  scenarios,  it  is  projected  that  the  workload  associated 
with  Data  Link  operations  may  be  even  lower  than  that  experienced 
with  current  voice  communication  procedures  when  the  system  is  used 
by  fully  experienced  controllers  in  familiar  field  environments. 

f.  Controllers  reported  no  negative  effects  of  extending  Data 
Link  transmission  delays  beyond  those  currently  projected  for  an 
operational  Mode  S  system. 

g.  Controller  workload  did  not  differ  between  the  20  and  the 
70  percent  Data  Link  aircraft  equipage  tests.  Furthermore, 
controllers  reported  no  significant  problems  in  switching  between 
voice  and  Data  Link  in  mixed  equipage  environments.  These  results 
suggest  that  Data  Link  should  be  equally  usable  by  field 
controllers  in  early  and  later  phases  of  Data  Link  system 
implementation . 

h.  The  tested  designs  of  the  four  Data  Link  services  were 
defined  by  the  results  of  preliminary  mini  studies.  The 
operational  evaluation  demonstrated  that  these  en  route  controller 
procedures,  displays,  and  data  entry  protocols  were  sufficiently 
effective  and  acceptable  to  controllers  to  warrant  their 
implementation  for  field  evaluation. 
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i.  As  noted  above,  the  quantitative  results  of  the  operational 
evaluation  did  not  reveal  major  deficiencies  which  would  prevent 
implementation  of  the  tested  Data  Link  services.  However, 
individual  controller  comments  and  the  group  discussion  sessions 
surfaced  some  open  issues  and  suggestions  for  enhancements  to  the 
current  service  designs  which  should  be  addressed  in  future  work. 
Those  topics  requiring  further  consideration  are  treated  in  the 
"Recommendations"  section  of  this  report. 

5.2  AIRCREW  SUBSTUDY. 


a.  The  operational  evaluation  study  represented  the  first  FAA 
investigation  of  aircrew  responses  to  Data  Link  communications 
using  preliminary  hardware  and  software  designs  for  pilot 
information  displays  and  controls.  Overall  pilot  perceptions  of  the 
Data  Link  concept  were  positive,  and  suggested  that  its 
implementation  would  enhance  the  quality  of  air-ground 
communications.  Evaluations  of  the  specific  methods  used  for 
presenting  Data  Link  information  in  this  study  indicated  that  minor 
changes  in  display  logic  and  format  would  enhance  interaction 
between  pilots  and  the  system. 

b.  Comparative  measures  indicated  that  the  time  required  by 
pilots  to  process  and  respond  to  ATC  instructions  was  shorter  when 
using  Data  Link  than  when  using  current  voice  radio  procedures. 
This  improvement  was  attributable  to  an  increased  rate  of 
information  transfer  as  misunderstandings  and  requirements  to  call¬ 
back  for  repetition  or  clarification  were  reduced. 

c.  Pilot  processing  time  for  weather  messages  were  also 
reduced  by  Data  Link  as  a  result  of  the  enhanced  interpretabil ity 
of  messages.  No  major  differences  between  voice  and  Data  Link  were 
detected  for  preparing  and  sending  weather  requests. 

d.  The  workload  associated  with  Data  Link  operations  was  rated 
as  acceptable  by  the  subject  pilots.  Data  Link  can  be  expected  to 
reduce  aircrew  workload  by  minimizing  memory  demands  and 
requirements  for  maintaining  notes  regarding  ATC  clearances. 
However,  effective  and  timely  crew  coordination  will  be  needed  with 
Data  Link  to  ensure  that  messages  reaching  one  crew  member  are 
communicated  to  others. 

e.  The  subject  pilots  in  this  study  concurred  that  Data  Link 
can  effectively  replace  most  functions  currently  served  by  voice 
radio  in  en  route  and  outer  terminal  control  environments.  Voice 
should  be  used  as  a  primary  back-up  to  Data  Link  and  as  a  channel 
for  emergency  communications. 

5.3  JOINT  ISSUES. 

Bused  on  the  results  of  the  operational  evaluation  and  those  of 
earlier  investigations,  it  appears  that  pilots  and  controllers 
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agree  on  a  number  of  basic  procedures  which  will  govern  air-ground 
interactions  via  Data  Link.  However,  additional  formal  meetings 
between  pilot  and  controller  groups  will  be  necessary  to  resolve 
other  outstanding  issues.  These  issues  include  elimination  of  the 
initial  call  after  a  transfer  of  communication,  procedures  for 
communication  when  a  pilot  does  not  confirm  receipt  of  an  ATC 
clearance,  and  compensation  for  the  potential  loss  of  pilot 
situation  awareness  which  is  currently  supported  by  monitoring 
broadcasted  radio  calls  to  other  aircraft. 

6 .  RECOMMENDATIONS . 

The  following  recommendations  for  future  efforts  under  the  Federal 
Aviation  Administration  (FAA)  Data  Link  program  are  derived  from 
the  findings  and  conclusions  of  the  present  research: 

a.  This  operational  evaluation  study  demonstrated  that  the 
tested  Data  Link  ATC  services  and  functions  can  significantly 
reduce  ground-air  radio  frequency  congestion  without  impairing  air 
traffic  control  (ATC)  safety  or  the  performance  capabilities  of  air 
traffic  controllers.  Therefore,  it  is  recommended  that  these 
services  be  incorporated  as  modifications  to  current  en  route  ATC 
software  and  hardware,  and  that  they  be  subjected  to  operational 
test  and  evaluation  (OT&E) . 

b.  It  is  recommended  that  the  following  outstanding  issues 
and  potential  enhancements  to  the  current  Data  Link  service  designs 
be  addressed  by  the  FAA. 

1.  Available  graphic  symbologies  for  display  of  Data  Link 
equipage  and  uplink  eligibility  in  the  full  data  block  ( FDB)  should 
be  reviewed  to  identify  alternatives  which  could  enhance  the 
perceptual  discrimination  and  detectability  of  this  information  by 
air  traffic  controllers. 

2.  The  persistence  of  the  FDB  display  of  a  downlinked 
WILCO  response  should  be  reconsidered  to  determine  whether  an 
increased  display  time  will  reduce  the  demands  of  monitoring  Data 
Link  transactions. 

3.  The  list  display  of  transaction  status  should  be 
examined  to  determine  any  requirement  to  reduce  display  clutter  by 
creating  a  separate  list  for  transfer  of  communication 
transactions,  or  by  displaying  information  regarding  only  those 
transactions  which  have  not  been  successfully  completed. 

4.  An  option  which  would  permit  automatic  uplinks  of  the 
transfer  of  communication  service  should  be  reviewed. 

5.  Uplinks  of  requested  altitudes  using  menu  text  should 
be  evaluated  to  determine  whether  this  type  of  clearance  should  be 
accessed  by  selecting  a  separate  "hard"  altitude  menu. 
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6.  Menu  text  should  be  examined  to  consider  the  need  for 
inclusion  of  extended,  compound  clearances  as  menu  items. 

7.  Proposed  operational  procedures  should  be  developed  to 
govern  the  conditions  under  which  the  free  text  service  will  be 
used,  and  to  define  eligibility  rules  for  engaging  in  Data  Link 
communications  while  a  transfer  of  communication  transaction  is  in 
progress . 

8.  Proposed  procedures  for  assigning  Data  Link  duties  to 
associate  (D)  controllers  should  be  identified.  These  should 
include  any  software  modifications  that  would  be  needed  to  provide 
Data  Link  control  to  the  appropriate  workstation  position. 

c.  It  is  recommended  that  the  topics  outlined  above,  as  well 
as  any  additional  outstanding  issues,  be  addressed  through  formal 
presentation  to  the  Data  Link  Validation  Team  (ATDLVT)  .  The 
results  of  the  deliberations  of  this  team  should  define  any 
requirements  for  additional  research  to  refine  the  designs  of  the 
initial  Data  Link  services. 

d.  Based  on  the  success  of  the  operational  evaluation  and  the 
comments  of  the  participating  controllers  and  pilots,  it  is 
recommended  that  research  and  development  work  be  expedited  to 
extend  Data  Link  applications  to  the  full  range  of  ATC  services. 

e.  The  existence  of  outstanding  questions  regarding  the  rules 
and  procedures  which  will  govern  pilot  and  controller  interactions 
using  Data  Link  dictates  that  a  mechanism  be  developed  for 
resolving  these  issues.  Therefore,  it  is  recommended  that  formal 
meetings  between  the  ATDLVT  and  a  representative  pilot  group  be 
instituted  to  address  joint  issues  and  to  suggest  direction  for 
additional  relevant  research. 

f.  Based  on  the  positive  findings  of  the  aircrew  substudy,  it 
is  recommended  that  Data  Link  services  to  aircraft  be  implemented 
in  the  National  Airspace  System  (NAS)  as  rapidly  as  possible. 

g.  Because  the  design  and  development  of  airborne  Data  Link 
procedures  and  crew  interfaces  has  lagged  behind  corresponding  work 
with  ground  based  air  traffic  control  (ATC)  at  the  FAA  Technical 
Center,  it  is  recommended  that  additional  design  studies  and  end- 
to-end  operational  evaluations  be  conducted  to  address  the 
following  requirements: 

1.  Research  is  required  to  investigate  methods  for 
integration  of  Data  Link  into  the  cockpit  environment.  This 
research  should  address  the  physical  location  of  cockpit  displays 
and  controls,  integration  of  Data  Link  with  other  airborne  systems 
(e.g.,  navigation),  and  integration  of  Data  Link  tasks  with  other 
flight  duties  to  minimize  the  number  and  complexity  of  required 
activities . 
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2.  Additional  research  is  also  required  to  evaluate  and 
refine  alternative  display  and  control  hardware  for  Data  Link;  and 
to  modify  display  formats  and  logic  in  order  to  env  mce  information 
transfer  and  reduce  switching  requirements. 

3.  Additional  work  is  needed  to  analyze  experimentally 
the  observed  differences  in  performance  times  between  the  GAT 
pilots  and  the  B-727  crews. 

4.  Additional  work  is  needed  to  assess  the  impact  of 
realistic  crew  response  times  and  Data  Link  transmission  times  on 
ATC  system  performance.  More  complex  flight  scenarios  and  more 
pilot  participation  in  the  simulated  traffic  scenarios  will  be 
needed. 
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